{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 利用线性回归技术实现 Ames 房价预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#数据说明： Ames 房价预测是 Kaggle 平台上的一个竞赛任务，需要根据房屋\n",
    "的特征来预测亚美尼亚州洛瓦市（Ames，Lowa）的房价。其中房屋的特征 x 共\n",
    "有 79 维，响应值 y 为每个房屋的销售价格（SalePrice）。评价标准为预测值的\n",
    "对数和观测值的对数的 RMSE(Root-Mean-Squared-Error )。\n",
    "#作业要求：\n",
    "1. 对数据做数据探索分析\n",
    "2. 适当的数据清洗（异常值处理和缺失值处理）\n",
    "3. 适当的特征工程\n",
    "4. 用线性回归模型对房价进行预测（最小二乘、岭回归、Lasso），注意正则超\n",
    "参数的调优。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "#导入工具包\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn import linear_model, discriminant_analysis\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数据探索及准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "具体的数据探索的方法\n",
    "数据探索和数据准备的六大步骤。\n",
    "1. 变量的识别\n",
    "2. 单变量的分析\n",
    "3. 双变量的分析\n",
    "4. 处理缺失值\n",
    "5. 处理异常值\n",
    "6. 特征提取(Feature Engineering)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "#缺失值处理\n",
    "def process_missvalue_by_meaning(df):\n",
    "    # Alley : data description says NA means \"no alley access\"\n",
    "    df.loc[:, \"Alley\"] = df.loc[:, \"Alley\"].fillna(\"None\")\n",
    "    # BedroomAbvGr : NA most likely means 0\n",
    "    df.loc[:, \"BedroomAbvGr\"] = df.loc[:, \"BedroomAbvGr\"].fillna(0)\n",
    "    # BsmtQual etc : data description says NA for basement features is \"no\n",
    "    # basement\"\n",
    "    df.loc[:, \"BsmtQual\"] = df.loc[:, \"BsmtQual\"].fillna(\"No\")\n",
    "    df.loc[:, \"BsmtCond\"] = df.loc[:, \"BsmtCond\"].fillna(\"No\")\n",
    "    df.loc[:, \"BsmtExposure\"] = df.loc[:, \"BsmtExposure\"].fillna(\"No\")\n",
    "    df.loc[:, \"BsmtFinType1\"] = df.loc[:, \"BsmtFinType1\"].fillna(\"No\")\n",
    "    df.loc[:, \"BsmtFinType2\"] = df.loc[:, \"BsmtFinType2\"].fillna(\"No\")\n",
    "    df.loc[:, \"BsmtFullBath\"] = df.loc[:, \"BsmtFullBath\"].fillna(0)\n",
    "    df.loc[:, \"BsmtHalfBath\"] = df.loc[:, \"BsmtHalfBath\"].fillna(0)\n",
    "    df.loc[:, \"BsmtUnfSF\"] = df.loc[:, \"BsmtUnfSF\"].fillna(0)\n",
    "    # CentralAir : NA most likely means No\n",
    "    df.loc[:, \"CentralAir\"] = df.loc[:, \"CentralAir\"].fillna(\"N\")\n",
    "    # Condition : NA most likely means Normal，靠近主干道或铁路\n",
    "    df.loc[:, \"Condition1\"] = df.loc[:, \"Condition1\"].fillna(\"Norm\")\n",
    "    df.loc[:, \"Condition2\"] = df.loc[:, \"Condition2\"].fillna(\"Norm\")\n",
    "    # EnclosedPorch : NA most likely means no enclosed porch\n",
    "    df.loc[:, \"EnclosedPorch\"] = df.loc[:, \"EnclosedPorch\"].fillna(0)\n",
    "    # External stuff : NA most likely means average\n",
    "    df.loc[:, \"ExterCond\"] = df.loc[:, \"ExterCond\"].fillna(\"TA\")\n",
    "    df.loc[:, \"ExterQual\"] = df.loc[:, \"ExterQual\"].fillna(\"TA\")\n",
    "    # Fence : data description says NA means \"no fence\"\n",
    "    df.loc[:, \"Fence\"] = df.loc[:, \"Fence\"].fillna(\"No\")\n",
    "    # FireplaceQu : data description says NA means \"no fireplace\"\n",
    "    df.loc[:, \"FireplaceQu\"] = df.loc[:, \"FireplaceQu\"].fillna(\"No\")\n",
    "    df.loc[:, \"Fireplaces\"] = df.loc[:, \"Fireplaces\"].fillna(0)\n",
    "    # Functional : data description says NA means typical，家用（Home）功能性评级\n",
    "    df.loc[:, \"Functional\"] = df.loc[:, \"Functional\"].fillna(\"Typ\")\n",
    "    # GarageType etc : data description says NA for garage features is \"no\n",
    "    # garage\"\n",
    "    df.loc[:, \"GarageType\"] = df.loc[:, \"GarageType\"].fillna(\"No\")\n",
    "    df.loc[:, \"GarageFinish\"] = df.loc[:, \"GarageFinish\"].fillna(\"No\")\n",
    "    df.loc[:, \"GarageQual\"] = df.loc[:, \"GarageQual\"].fillna(\"No\")\n",
    "    df.loc[:, \"GarageCond\"] = df.loc[:, \"GarageCond\"].fillna(\"No\")\n",
    "    df.loc[:, \"GarageArea\"] = df.loc[:, \"GarageArea\"].fillna(0)\n",
    "    df.loc[:, \"GarageCars\"] = df.loc[:, \"GarageCars\"].fillna(0)\n",
    "    # HalfBath : NA most likely means no half baths above grade\n",
    "    df.loc[:, \"HalfBath\"] = df.loc[:, \"HalfBath\"].fillna(0)\n",
    "    # HeatingQC : NA most likely means typical\n",
    "    df.loc[:, \"HeatingQC\"] = df.loc[:, \"HeatingQC\"].fillna(\"TA\")\n",
    "    # KitchenAbvGr : NA most likely means 0\n",
    "    df.loc[:, \"KitchenAbvGr\"] = df.loc[:, \"KitchenAbvGr\"].fillna(0)\n",
    "    # KitchenQual : NA most likely means typical\n",
    "    df.loc[:, \"KitchenQual\"] = df.loc[:, \"KitchenQual\"].fillna(\"TA\")\n",
    "    # LotFrontage : NA most likely means no lot frontage\n",
    "    df.loc[:, \"LotFrontage\"] = df.loc[:, \"LotFrontage\"].fillna(0)\n",
    "    # LotShape : NA most likely means regular\n",
    "    df.loc[:, \"LotShape\"] = df.loc[:, \"LotShape\"].fillna(\"Reg\")\n",
    "    # MasVnrType : NA most likely means no veneer，表层砌体（Masonry veneer）类型\n",
    "    df.loc[:, \"MasVnrType\"] = df.loc[:, \"MasVnrType\"].fillna(\"None\")\n",
    "    df.loc[:, \"MasVnrArea\"] = df.loc[:, \"MasVnrArea\"].fillna(0)\n",
    "    # MiscFeature : data description says NA means \"no misc feature\"\n",
    "    df.loc[:, \"MiscFeature\"] = df.loc[:, \"MiscFeature\"].fillna(\"No\")\n",
    "    df.loc[:, \"MiscVal\"] = df.loc[:, \"MiscVal\"].fillna(0)\n",
    "    # OpenPorchSF : NA most likely means no open porch\n",
    "    df.loc[:, \"OpenPorchSF\"] = df.loc[:, \"OpenPorchSF\"].fillna(0)\n",
    "    # PavedDrive : NA most likely means not paved\n",
    "    df.loc[:, \"PavedDrive\"] = df.loc[:, \"PavedDrive\"].fillna(\"N\")\n",
    "    # PoolQC : data description says NA means \"no pool\"\n",
    "    df.loc[:, \"PoolQC\"] = df.loc[:, \"PoolQC\"].fillna(\"No\")\n",
    "    df.loc[:, \"PoolArea\"] = df.loc[:, \"PoolArea\"].fillna(0)\n",
    "    # SaleCondition : NA most likely means normal sale\n",
    "    df.loc[:, \"SaleCondition\"] = df.loc[:, \"SaleCondition\"].fillna(\"Normal\")\n",
    "    # ScreenPorch : NA most likely means no screen porch，观景门廊\n",
    "    df.loc[:, \"ScreenPorch\"] = df.loc[:, \"ScreenPorch\"].fillna(0)\n",
    "    # TotRmsAbvGrd : NA most likely means 0\n",
    "    df.loc[:, \"TotRmsAbvGrd\"] = df.loc[:, \"TotRmsAbvGrd\"].fillna(0)\n",
    "    # Utilities : NA most likely means all public utilities\n",
    "    df.loc[:, \"Utilities\"] = df.loc[:, \"Utilities\"].fillna(\"AllPub\")\n",
    "    # WoodDeckSF : NA most likely means no wood deck\n",
    "    df.loc[:, \"WoodDeckSF\"] = df.loc[:, \"WoodDeckSF\"].fillna(0)\n",
    "\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "#数据值转换\n",
    "def numberical2cat(df):\n",
    "    df.replace({\"MSSubClass\": {20: \"SC20\", 30: \"SC30\", 40: \"SC40\", 45: \"SC45\",\n",
    "                               50: \"SC50\", 60: \"SC60\", 70: \"SC70\", 75: \"SC75\",\n",
    "                               80: \"SC80\", 85: \"SC85\", 90: \"SC90\", 120: \"SC120\",\n",
    "                               150: \"SC150\", 160: \"SC160\", 180: \"SC180\", 190: \"SC190\"},\n",
    "                \"MoSold\": {1: \"Jan\", 2: \"Feb\", 3: \"Mar\", 4: \"Apr\", 5: \"May\", 6: \"Jun\",\n",
    "                           7: \"Jul\", 8: \"Aug\", 9: \"Sep\", 10: \"Oct\", 11: \"Nov\", 12: \"Dec\"}\n",
    "                }, inplace=True)\n",
    "\n",
    "    return df\n",
    "def cat2numberical(df):\n",
    "    df.replace({\"Alley\": {\"None\": 0, \"Grvl\": 1, \"Pave\": 2},\n",
    "                \"BsmtCond\": {\"No\": 0, \"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"BsmtExposure\": {\"No\": 0, \"Mn\": 1, \"Av\": 2, \"Gd\": 3},\n",
    "                \"BsmtFinType1\": {\"No\": 0, \"Unf\": 1, \"LwQ\": 2, \"Rec\": 3, \"BLQ\": 4,\n",
    "                                 \"ALQ\": 5, \"GLQ\": 6},\n",
    "                \"BsmtFinType2\": {\"No\": 0, \"Unf\": 1, \"LwQ\": 2, \"Rec\": 3, \"BLQ\": 4,\n",
    "                                 \"ALQ\": 5, \"GLQ\": 6},\n",
    "                \"BsmtQual\": {\"No\": 0, \"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"ExterCond\": {\"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"ExterQual\": {\"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"FireplaceQu\": {\"No\": 0, \"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"Functional\": {\"Sal\": 1, \"Sev\": 2, \"Maj2\": 3, \"Maj1\": 4, \"Mod\": 5,\n",
    "                               \"Min2\": 6, \"Min1\": 7, \"Typ\": 8},\n",
    "                \"GarageCond\": {\"No\": 0, \"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"GarageQual\": {\"No\": 0, \"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"HeatingQC\": {\"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"KitchenQual\": {\"Po\": 1, \"Fa\": 2, \"TA\": 3, \"Gd\": 4, \"Ex\": 5},\n",
    "                \"LandSlope\": {\"Sev\": 1, \"Mod\": 2, \"Gtl\": 3},\n",
    "                \"LotShape\": {\"IR3\": 1, \"IR2\": 2, \"IR1\": 3, \"Reg\": 4},\n",
    "                \"PavedDrive\": {\"N\": 0, \"P\": 1, \"Y\": 2},\n",
    "                \"PoolQC\": {\"No\": 0, \"Fa\": 1, \"TA\": 2, \"Gd\": 3, \"Ex\": 4},\n",
    "                \"Street\": {\"Grvl\": 1, \"Pave\": 2},\n",
    "                \"Utilities\": {\"ELO\": 1, \"NoSeWa\": 2, \"NoSewr\": 3, \"AllPub\": 4}},\n",
    "               inplace=True\n",
    "               )\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "#特征提取\n",
    "#合并类别\n",
    "def simplify(df):\n",
    "    df[\"SimplOverallQual\"] = df.OverallQual.replace({1: 1, 2: 1, 3: 1,  # bad\n",
    "                                                     4: 2, 5: 2, 6: 2,  # average\n",
    "                                                     7: 3, 8: 3, 9: 3, 10: 3  # good\n",
    "                                                     }, inplace=True)\n",
    "    df[\"SimplOverallCond\"] = df.OverallCond.replace({1: 1, 2: 1, 3: 1,  # bad\n",
    "                                                     4: 2, 5: 2, 6: 2,  # average\n",
    "                                                     7: 3, 8: 3, 9: 3, 10: 3  # good\n",
    "                                                     }, inplace=True)\n",
    "    df[\"SimplPoolQC\"] = df.PoolQC.replace({1: 1, 2: 1,  # average\n",
    "                                           3: 2, 4: 2  # good\n",
    "                                           }, inplace=True)\n",
    "    df[\"SimplGarageCond\"] = df.GarageCond.replace({1: 1,  # bad\n",
    "                                                   2: 1, 3: 1,  # average\n",
    "                                                   4: 2, 5: 2  # good\n",
    "                                                   }, inplace=True)\n",
    "    df[\"SimplGarageQual\"] = df.GarageQual.replace({1: 1,  # bad\n",
    "                                                   2: 1, 3: 1,  # average\n",
    "                                                   4: 2, 5: 2  # good\n",
    "                                                   }, inplace=True)\n",
    "    df[\"SimplFireplaceQu\"] = df.FireplaceQu.replace({1: 1,  # bad\n",
    "                                                     2: 1, 3: 1,  # average\n",
    "                                                     4: 2, 5: 2  # good\n",
    "                                                     }, inplace=True)\n",
    "    df[\"SimplFireplaceQu\"] = df.FireplaceQu.replace({1: 1,  # bad\n",
    "                                                     2: 1, 3: 1,  # average\n",
    "                                                     4: 2, 5: 2  # good\n",
    "                                                     }, inplace=True)\n",
    "    df[\"SimplFunctional\"] = df.Functional.replace({1: 1, 2: 1,  # bad\n",
    "                                                   3: 2, 4: 2,  # major\n",
    "                                                   5: 3, 6: 3, 7: 3,  # minor\n",
    "                                                   8: 4  # typical\n",
    "                                                   }, inplace=True)\n",
    "    df[\"SimplKitchenQual\"] = df.KitchenQual.replace({1: 1,  # bad\n",
    "                                                     2: 1, 3: 1,  # average\n",
    "                                                     4: 2, 5: 2  # good\n",
    "                                                     }, inplace=True)\n",
    "    df[\"SimplHeatingQC\"] = df.HeatingQC.replace({1: 1,  # bad\n",
    "                                                 2: 1, 3: 1,  # average\n",
    "                                                 4: 2, 5: 2  # good\n",
    "                                                 }, inplace=True)\n",
    "    df[\"SimplBsmtFinType1\"] = df.BsmtFinType1.replace({1: 1,  # unfinished\n",
    "                                                       2: 1, 3: 1,  # rec room\n",
    "                                                       4: 2, 5: 2, 6: 2  # living quarters\n",
    "                                                       }, inplace=True)\n",
    "    df[\"SimplBsmtFinType2\"] = df.BsmtFinType2.replace({1: 1,  # unfinished\n",
    "                                                       2: 1, 3: 1,  # rec room\n",
    "                                                       4: 2, 5: 2, 6: 2  # living quarters\n",
    "                                                       }, inplace=True)\n",
    "    df[\"SimplBsmtCond\"] = df.BsmtCond.replace({1: 1,  # bad\n",
    "                                               2: 1, 3: 1,  # average\n",
    "                                               4: 2, 5: 2  # good\n",
    "                                               }, inplace=True)\n",
    "    df[\"SimplBsmtQual\"] = df.BsmtQual.replace({1: 1,  # bad\n",
    "                                               2: 1, 3: 1,  # average\n",
    "                                               4: 2, 5: 2  # good\n",
    "                                               }, inplace=True)\n",
    "    df[\"SimplExterCond\"] = df.ExterCond.replace({1: 1,  # bad\n",
    "                                                 2: 1, 3: 1,  # average\n",
    "                                                 4: 2, 5: 2  # good\n",
    "                                                 }, inplace=True)\n",
    "    df[\"SimplExterQual\"] = df.ExterQual.replace({1: 1,  # bad\n",
    "                                                 2: 1, 3: 1,  # average\n",
    "                                                 4: 2, 5: 2  # good\n",
    "                                                 }, inplace=True)\n",
    "    return df\n",
    "def Combine(df):\n",
    "    # Overall quality of the house\n",
    "    df[\"OverallGrade\"] = df[\"OverallQual\"] * df[\"OverallCond\"]\n",
    "    # Overall quality of the garage\n",
    "    df[\"GarageGrade\"] = df[\"GarageQual\"] * df[\"GarageCond\"]\n",
    "    # Overall quality of the exterior\n",
    "    df[\"ExterGrade\"] = df[\"ExterQual\"] * df[\"ExterCond\"]\n",
    "    # Overall kitchen score\n",
    "    df[\"KitchenScore\"] = df[\"KitchenAbvGr\"] * df[\"KitchenQual\"]\n",
    "    # Overall fireplace score\n",
    "    df[\"FireplaceScore\"] = df[\"Fireplaces\"] * df[\"FireplaceQu\"]\n",
    "    # Overall garage score\n",
    "    df[\"GarageScore\"] = df[\"GarageArea\"] * df[\"GarageQual\"]\n",
    "    # Overall pool score\n",
    "    df[\"PoolScore\"] = df[\"PoolArea\"] * df[\"PoolQC\"]\n",
    "    # Simplified overall quality of the house\n",
    "    df[\"SimplOverallGrade\"] = df[\"SimplOverallQual\"] * df[\"SimplOverallCond\"]\n",
    "    # Simplified overall quality of the exterior\n",
    "    df[\"SimplExterGrade\"] = df[\"SimplExterQual\"] * df[\"SimplExterCond\"]\n",
    "    # Simplified overall pool score\n",
    "    df[\"SimplPoolScore\"] = df[\"PoolArea\"] * df[\"SimplPoolQC\"]\n",
    "    # Simplified overall garage score\n",
    "    df[\"SimplGarageScore\"] = df[\"GarageArea\"] * df[\"SimplGarageQual\"]\n",
    "    # Simplified overall fireplace score\n",
    "    df[\"SimplFireplaceScore\"] = df[\"Fireplaces\"] * df[\"SimplFireplaceQu\"]\n",
    "    # Simplified overall kitchen score\n",
    "    df[\"SimplKitchenScore\"] = df[\"KitchenAbvGr\"] * df[\"SimplKitchenQual\"]\n",
    "    # Total number of bathrooms\n",
    "    df[\"TotalBath\"] = df[\"BsmtFullBath\"] + (0.5 * df[\"BsmtHalfBath\"]) + \\\n",
    "        df[\"FullBath\"] + (0.5 * df[\"HalfBath\"])\n",
    "    # Total SF for house (incl. basement)\n",
    "    df[\"AllSF\"] = df[\"GrLivArea\"] + df[\"TotalBsmtSF\"]\n",
    "    # Total SF for 1st + 2nd floors\n",
    "    df[\"AllFlrsSF\"] = df[\"1stFlrSF\"] + df[\"2ndFlrSF\"]\n",
    "    # Total SF for porch\n",
    "    df[\"AllPorchSF\"] = df[\"OpenPorchSF\"] + df[\"EnclosedPorch\"] + \\\n",
    "        df[\"3SsnPorch\"] + df[\"ScreenPorch\"]\n",
    "    # Has masonry veneer or not\n",
    "    df[\"HasMasVnr\"] = df.MasVnrType.replace({\"BrkCmn\": 1, \"BrkFace\": 1, \"CBlock\": 1,\n",
    "                                             \"Stone\": 1, \"None\": 0})\n",
    "    # House completed before sale or not\n",
    "    df[\"BoughtOffPlan\"] = df.SaleCondition.replace({\"Abnorml\": 0, \"Alloca\": 0, \"AdjLand\": 0,\n",
    "                                                    \"Family\": 0, \"Normal\": 0, \"Partial\": 1})\n",
    "    return df\n",
    "def Polynomials_top10(df, top10_cols):\n",
    "    for i in range(1, 11):\n",
    "        new_cols_2 = top10_cols[0][i] + '_s' + str(2)\n",
    "        new_cols_3 = top10_cols[0][i] + '_s' + str(3)\n",
    "        new_cols_sq = top10_cols[0][i] + '_sq'\n",
    "\n",
    "        df[new_cols_2] = df[top10_cols[0][i]] ** 2\n",
    "        df[new_cols_3] = df[top10_cols[0][i]] ** 3\n",
    "        df[new_cols_sq] = np.sqrt(df[top10_cols[0][i]])\n",
    "\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对训练集的其他数值型特征进行空缺值填补（中值填补）\n",
    "# 返回填补后的dataframe，以及每列的中值，用于填补测试集的空缺值\n",
    "# 数值型特征还要进行数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "\n",
    "def fillna_numerical_train(df):\n",
    "    numerical_features = df.select_dtypes(exclude=[\"object\"]).columns\n",
    "\n",
    "    numerical_features = numerical_features.drop(\"SalePrice\")\n",
    "    print(\"Numerical features : \" + str(len(numerical_features)))\n",
    "\n",
    "    df.info()\n",
    "    df_num = df[numerical_features]\n",
    "    # df_num.info()\n",
    "\n",
    "    medians = df_num.median()\n",
    "    # Handle remaining missing values for numerical features by using median\n",
    "    # as replacement\n",
    "    print(\"NAs for numerical features in df : \" +\n",
    "          str(df_num.isnull().values.sum()))\n",
    "    df_num = df_num.fillna(medians)\n",
    "    print(\"Remaining NAs for numerical features in df : \" +\n",
    "          str(df_num.isnull().values.sum()))\n",
    "\n",
    "    # df_num.info()\n",
    "    # 分别初始化对特征和目标值的标准化器\n",
    "    ss_X = StandardScaler()\n",
    "\n",
    "    # 对训练特征进行标准化处理\n",
    "    temp = ss_X.fit_transform(df_num)\n",
    "    df_num = pd.DataFrame(\n",
    "        data=temp, columns=numerical_features, index=df_num.index)\n",
    "\n",
    "    return df_num, medians, ss_X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对测试集的其他数值型特征进行空缺值填补（用训练集中相应列的中值填补）\n",
    "def fillna_numerical_test(df, medians, ss_X):\n",
    "    numerical_features = df.select_dtypes(exclude=[\"object\"]).columns\n",
    "    # numerical_features = numerical_features.drop(\"SalePrice\")\n",
    "    # #测试集中没有SalePrice\n",
    "    print(\"Numerical features : \" + str(len(numerical_features)))\n",
    "    df_num = df[numerical_features]\n",
    "    # Handle remaining missing values for numerical features by using median\n",
    "    # as replacement\n",
    "    print(\"NAs for numerical features in df : \" +\n",
    "          str(df_num.isnull().values.sum()))\n",
    "    df_num = df_num.fillna(medians)\n",
    "    print(\"Remaining NAs for numerical features in df : \" +\n",
    "          str(df_num.isnull().values.sum()))\n",
    "    # 对数值特征进行标准化\n",
    "    temp = ss_X.transform(df_num)\n",
    "    df_num = pd.DataFrame(\n",
    "        data=temp, columns=numerical_features, index=df_num.index)\n",
    "    return df_num\n",
    "\n",
    "\n",
    "def get_dummies_cat(df):\n",
    "    categorical_features = df.select_dtypes(include=[\"object\"]).columns\n",
    "    print(\"Categorical features : \" + str(len(categorical_features)))\n",
    "    df_cat = df[categorical_features]\n",
    "    # Create dummy features for categorical values via one-hot encoding\n",
    "    print(\"NAs for categorical features in df : \" +\n",
    "          str(df_cat.isnull().values.sum()))\n",
    "    df_cat = pd.get_dummies(df_cat, dummy_na=True)\n",
    "    print(\"Remaining NAs for categorical features in df : \" +\n",
    "          str(df_cat.isnull().values.sum()))\n",
    "    return df_cat\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_data():\n",
    "    # 读取文件数据\n",
    "    train = pd.read_csv(\"Ames_House_train.csv\")\n",
    "    test = pd.read_csv(\"Ames_House_test.csv\")\n",
    "    # 缺失值处理\n",
    "    train = process_missvalue_by_meaning(train)\n",
    "    test = process_missvalue_by_meaning(test)\n",
    "    #数据值转换\n",
    "    train = numberical2cat(train)\n",
    "    test = numberical2cat(test)\n",
    "    train = cat2numberical(train)\n",
    "    test = cat2numberical(test)\n",
    "    # 合并类别\n",
    "    train = simplify(train)\n",
    "    test = simplify(test)\n",
    "    train = Combine(train)\n",
    "    test = Combine(test)\n",
    "    #\n",
    "    corr = train.corr()\n",
    "    corr.sort_values([\"SalePrice\"], ascending=False, inplace=True)\n",
    "    # the first one is SalePrice itself,from 1-11\n",
    "    threshold = corr.SalePrice.iloc[11]\n",
    "    top10_cols = (corr.SalePrice[corr['SalePrice'] > threshold]).axes\n",
    "    #\n",
    "    train = Polynomials_top10(train, top10_cols)\n",
    "    test = Polynomials_top10(test, top10_cols)\n",
    "    # 对训练集的其他数值型特征进行空缺值填补（中值填补）\n",
    "    # 返回填补后的dataframe，以及每列的中值，用于填补测试集的空缺值\n",
    "    # 数值型特征还要进行数据标准化\n",
    "    train_num, medians, ss_X = fillna_numerical_train(train)\n",
    "    test_num = fillna_numerical_test(test, medians, ss_X)\n",
    "    n_train_samples = train.shape[0]\n",
    "    train_test = pd.concat((train, test), axis=0)\n",
    "    train_test_cat = get_dummies_cat(train_test)\n",
    "    train_cat = train_test_cat.iloc[:n_train_samples, :]\n",
    "    test_cat = train_test_cat.iloc[n_train_samples:, :]\n",
    "    # 分离建模的特征变量和目标变量\n",
    "    data = train.select_dtypes(include=[np.number]).interpolate().dropna()\n",
    "    sum(data.isnull().sum() != 0)\n",
    "    y = np.log(train.SalePrice)\n",
    "    X = data.drop(['SalePrice', 'Id'], axis=1)\n",
    "    X_train, X_test, y_train, y_test = train_test_split(\n",
    "        X, y, random_state=42, test_size=.33)\n",
    "    return X_train, X_test, y_train, y_test,train, test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Numerical features : 98\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1460 entries, 0 to 1459\n",
      "Columns: 145 entries, Id to 1stFlrSF_sq\n",
      "dtypes: float64(16), int64(83), object(46)\n",
      "memory usage: 1.6+ MB\n",
      "NAs for numerical features in df : 81\n",
      "Remaining NAs for numerical features in df : 0\n",
      "Numerical features : 98\n",
      "NAs for numerical features in df : 88\n",
      "Remaining NAs for numerical features in df : 0\n",
      "Categorical features : 46\n",
      "NAs for categorical features in df : 61307\n",
      "Remaining NAs for categorical features in df : 0\n"
     ]
    }
   ],
   "source": [
    "#获得数据集\n",
    "X_train,X_test,y_train,y_test,train,test = load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 用线性回归模型对房价进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "权重向量:[  1.88338193e-04   1.37966140e-06   1.97379723e-01   1.07748634e-02\n",
      "  -1.84689017e-02   5.31544547e-02  -1.61669076e-02   4.00190798e-02\n",
      "   1.02421888e-01   2.25206513e-03   1.23223086e-03   9.95838207e-05\n",
      "   1.54485488e-04  -7.23608747e-02   2.28702588e-02  -2.95145822e-03\n",
      "   1.42722927e-02   5.58037870e-03   4.63893380e-04  -6.55558871e-03\n",
      "   3.91414715e-04   3.69428519e-04   1.22473675e-03   2.74356251e-02\n",
      "  -5.44415280e-03   1.19624660e-03   3.12209652e-03  -1.12580963e-03\n",
      "  -4.81639352e-01  -2.49364998e-01  -4.77423600e-01  -2.19716423e-01\n",
      "  -1.10368151e-02  -1.59679823e-01   1.34308382e-02   9.64812722e-03\n",
      "   8.08805760e-02   1.89524962e-02   1.38718970e-02  -3.36738026e-04\n",
      "   1.42315535e+00  -7.64479271e-03   9.92300217e-03  -8.55308389e-02\n",
      "   1.96586917e-02   5.70637647e-05  -1.23911175e-04   3.89610388e-05\n",
      "   2.03438542e-05   1.82583895e-04  -7.69176261e-04  -7.19096761e-01\n",
      "  -7.24221508e-06   5.28669365e-03   1.43954772e-03   6.76814031e-02\n",
      "   3.53238962e-02   6.84195930e-03   1.89524962e-02  -8.19315332e-05\n",
      "   1.49015606e-03  -1.19360366e+00   9.89271788e-05  -4.24790617e-03\n",
      "   1.17977585e-04  -2.82120531e-02   8.51457519e-02  -1.07048775e-08\n",
      "  -8.76779552e-12  -5.54551674e-02   1.52558863e-06  -1.93590387e-10\n",
      "   2.29970378e-01  -6.69931729e-07   1.19075375e-10   5.35935350e-03\n",
      "   7.05114880e-02  -1.71419061e-02   1.89640127e-02  -2.10431945e-01\n",
      "   1.77070599e-02  -2.13185205e+00   1.93091756e-01  -1.07854144e-02\n",
      "   2.89214395e+00   5.44068417e-06  -1.91399729e-09   1.68769545e-01\n",
      "  -1.04861319e-03  -2.44676410e-03  -1.44783268e-04  -3.81662841e-07\n",
      "   9.95669797e-11  -1.92222904e-02   1.49128301e-06  -1.86209187e-10\n",
      "   2.66193422e-01], b的值为:-13.40\n",
      "损失函数的值: 0.02\n",
      "预测性能得分: 0.85\n"
     ]
    }
   ],
   "source": [
    "def test_LinearRegression(*data):\n",
    "    X_train, X_test, y_train, y_test,train,test= data\n",
    "    #通过sklearn的linear_model创建线性回归对象\n",
    "    linearRegression = linear_model.LinearRegression()\n",
    "    #进行训练\n",
    "    linearRegression.fit(X_train, y_train)\n",
    "    #通过LinearRegression的coef_属性获得权重向量,intercept_获得b的值\n",
    "    print(\"权重向量:%s, b的值为:%.2f\" % (linearRegression.coef_, linearRegression.intercept_))\n",
    "    #计算出损失函数的值\n",
    "    print(\"损失函数的值: %.2f\" % np.mean((linearRegression.predict(X_test) - y_test) ** 2))\n",
    "    #计算预测性能得分\n",
    "    print(\"预测性能得分: %.2f\" % linearRegression.score(X_test, y_test))\n",
    "    ##生成预测\n",
    "    submission = pd.DataFrame()\n",
    "    submission['Id'] = test.Id\n",
    "    #根据上面所做的模型，从测试数据中选择特性\n",
    "    feats = test.select_dtypes(\n",
    "            include=[np.number]).drop(['Id'], axis=1).interpolate()\n",
    "    #生成预测\n",
    "    predictions = linearRegression.predict(feats)\n",
    "    #预测转换成正确的形式，用np.exp()来做预测，因为之前已经取了对数。\n",
    "    final_predictions = np.exp(predictions)\n",
    "    submission['SalePrice'] = final_predictions\n",
    "    #创建预测\n",
    "    submission.to_csv('LinearRegressionPredictions.csv', index=False)\n",
    "    \n",
    "test_LinearRegression(X_train, X_test, y_train, y_test,train,test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 用岭回归模型对房价进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "权重向量:[  1.67319191e-04   1.41152466e-06   1.41413662e-01   1.21940262e-02\n",
      "  -1.82490673e-02   4.17028385e-02  -1.29921734e-02   3.30114818e-02\n",
      "   8.28848852e-02   2.25956055e-03   1.22665719e-03   1.02053295e-04\n",
      "  -1.62819305e-04  -5.96868852e-02   2.41743897e-02  -2.82080525e-03\n",
      "   1.49932909e-02   6.30634894e-03   2.79626641e-04  -7.28752002e-03\n",
      "   2.07416607e-04   1.86580060e-04   6.73623377e-04   2.65775124e-02\n",
      "  -2.75140763e-03   5.17750484e-04   1.21075765e-03  -1.02289896e-03\n",
      "   1.26515352e-02  -3.85774116e-03   1.34003577e-02   2.87017357e-02\n",
      "  -9.37702161e-03  -1.19369432e-01   4.34879784e-02   1.05251615e-02\n",
      "   8.05004701e-02   2.01730109e-02   1.17862573e-02  -1.93557708e-04\n",
      "   1.09533605e-03  -1.87893019e-03   1.40360951e-02  -3.81903540e-02\n",
      "   1.97377757e-02   5.59284222e-05  -1.12074730e-04   4.24964588e-06\n",
      "   3.13761003e-05   1.88643096e-04  -1.16181341e-03  -1.10808246e-01\n",
      "  -1.07887725e-05   5.48868476e-03   1.22298683e-02   4.71803283e-02\n",
      "   2.61529791e-02  -2.49648673e-02   2.01730109e-02  -4.23862811e-05\n",
      "   7.75622018e-04   3.84738901e-02  -3.49276469e-04  -2.23365713e-03\n",
      "   1.12194125e-04  -2.96608867e-02   8.36119681e-02   1.54297151e-07\n",
      "  -2.07351649e-11   9.44042518e-03   9.03529897e-07  -1.23455767e-10\n",
      "   9.12522675e-02  -5.17378148e-07   1.02696702e-10   5.20768881e-02\n",
      "   5.80546808e-02  -1.47982149e-02   1.56485429e-02   1.10757984e-02\n",
      "  -7.48597792e-04  -5.04273130e-03  -2.56458712e-02   4.03855925e-03\n",
      "   3.31819741e-02   1.81307614e-06  -6.85206911e-10   3.10872645e-02\n",
      "  -4.88457917e-04  -1.13973514e-03  -6.74419646e-05  -4.25241015e-07\n",
      "   1.10473846e-10  -2.04919822e-02   6.18801136e-07  -6.38696293e-11\n",
      "   1.42195094e-01], b的值为:-11.18\n",
      "损失函数的值:0.02\n",
      "预测性能得分: 0.86\n"
     ]
    }
   ],
   "source": [
    "def test_ridge(*data):\n",
    "    X_train, X_test, y_train, y_test,train,test= data\n",
    "    ridgeRegression = linear_model.Ridge()\n",
    "    ridgeRegression.fit(X_train, y_train)\n",
    "    print(\"权重向量:%s, b的值为:%.2f\" % (ridgeRegression.coef_, ridgeRegression.intercept_))\n",
    "    print(\"损失函数的值:%.2f\" % np.mean((ridgeRegression.predict(X_test) - y_test) ** 2))\n",
    "    print(\"预测性能得分: %.2f\" % ridgeRegression.score(X_test, y_test))\n",
    "    ##生成预测\n",
    "    submission = pd.DataFrame()\n",
    "    submission['Id'] = test.Id\n",
    "    #根据上面所做的模型，从测试数据中选择特性\n",
    "    feats = test.select_dtypes(\n",
    "            include=[np.number]).drop(['Id'], axis=1).interpolate()\n",
    "    #生成预测\n",
    "    predictions = ridgeRegression.predict(feats)\n",
    "    #预测转换成正确的形式，用np.exp()来做预测，因为之前已经取了对数。\n",
    "    final_predictions = np.exp(predictions)\n",
    "    submission['SalePrice'] = final_predictions\n",
    "    #创建预测\n",
    "    submission.to_csv('RidgeRegressionPredictions.csv', index=False)\n",
    "    \n",
    "test_ridge(X_train, X_test, y_train, y_test,train,test)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#测试不同的α值对预测性能的影响"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEaCAYAAAA7YdFPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VfWd//HXJxtZIAlLkCUrCsqi\nEomAFZfWDe3Ura1LlY4t6mgL005tZ+o87G+svz6my6+dWq1d1FZba1FGq9XWvdZdhLDKLrIlgBIg\nECBk//z+uBd6jQECuSfn5ub9fDzOI/ec8z3nfr5ewztn+15zd0RERI5WStgFiIhIz6YgERGRLlGQ\niIhIlyhIRESkSxQkIiLSJQoSERHpEgWJSEDM7Fdm9p1DrHczO647axIJguk5EpGjZ2brgWOAVmAP\n8Bwww933dGJbB0a6+5pAixQJmI5IRLruM+7eFxgPlAO3hlyPSLdSkIjEibt/ADxPJFAwswfN7Hv7\n15vZt8xsi5ltNrMvx25rZgPN7GkzqzOzeWb2PTN7I2b9CWb2opntMLNVZnZFd/VL5HAUJCJxYmaF\nwIXAx05VmdlU4JvAecBI4Nx2Te4B9gJDgH+OTvu3zQFeBP4IDAauBn5hZmPj3wuRI6cgEem6J81s\nN1AFbAX+q4M2VwAPuPtSd98L3L5/hZmlAp8F/svd6919OfC7mG3/CVjv7g+4e4u7LwAeBz4XTHdE\njoyCRKTrLnX3fsDZwAnAoA7aDCMSNPttiHldAKS1Wx/7ugSYZGY790/ANUSOXkRCpyARiRN3fxV4\nEPhxB6u3AEUx88Uxr2uAFqAwZlls2yrgVXfPj5n6uvvN8alcpGsUJCLxdSdwnpmNb7d8NnCdmY0x\ns2xiTn+5eyvwJ+B2M8s2sxOAL8Zs+xdglJlNM7P06HSqmY0OuC8inaIgEYkjd68Bfg98p93yZ4mE\nzMtELsa/3G7TGUAe8AHwEDALaIxuuxs4H7gK2Bxt80OgT1D9EDkSeiBRJAGZ2Q+BIe7+z4dtLBIy\nHZGIJIDocyInWcREYDrwRNh1iXRGWtgFiAgA/YiczhpG5BbinwB/DrUikU7SqS0REekSndoSEZEu\nUZCIiEiX9IprJIMGDfLS0tKwyxAR6VHmz5+/zd0LDteuVwRJaWkplZWVYZchItKjmNmGw7fSqS0R\nEekiBYmIiHSJgkRERLpEQSIiIl2iIBERkS5RkIiISJf0itt/JXm4O7X1zVTtqGfbnkZSU4yMtBQy\nUlNIj04ZaXbgdXpqZF1GWgrpqUZqimFmYXdDJKkoSCThNDS3UrWjnqraeqp27GPjjnqqdtSzcUc9\n1bX72NPYctT7NuNAuKSnWjR4UhiWl0XpoGxKB+VQNjCHsoIcSgbkkJWRGseeiSQnBYl0u9Y254O6\nhn+EQ/RnVW0kNGp2N36kfWZ6CkX9sykekM3kEQMp7J9F8YBsBudm0uZOU0sbza2RqanFD7yOzLfR\n1BpdFm3X1PrRbRqaW9m0cx8vr6xh257qj7z30LxMSgfmRAJmUDalA3MoG5RD8cBs+qQpZERAQSIB\nqW9qYc3WPf84oqiNHFVU7ahn0859NLf+Y9TpFIOheVkUDcji7FEFFA/IpujAlEVB3z7ddjpqd0Mz\n67fVs277XtZvi0zrtu/luaVbqK1vPtDODIbnZ1E2KOcjQXP8kFyG5WXq9Jn0KgoS6TJ3p7p2Hws2\n1rJgQy3zN9ayYstuWtv+ERb9s9MpHpDN2OF5TB03NBoWkSOLoXlZZKQlxn0f/TLTObEwjxML8z62\nbld984GAWRed1m/fy5OLNrG74R+n24bkZjKhpD/lxflMKOnP2GF5CdM/kSAoSOSINba0snRTXSQ0\nNtSyYGMtW6Ono7IzUhlflM/NZx3LuOF5BwKjX2Z6yFV3XV52OuOz8xlflP+R5e7Ojr1NrN++l6Wb\n6pgf/e/y13e3ANAnLYWTCvM4paQ/pxRHpoJ++rp1SR694outKioqXIM2Hr2tdQ0s2Fh74B/IpZvq\naGptA6BoQBYTivtH/wLvzwlD+pGWqr++AT6sazgQtvM31rJ0064Dp/RKBmYzobg/5SX9mVDcn+OH\n9CM1RafDJLGY2Xx3rzhsOwWJxGppbWPlB7sPHGnM31BLde0+ADLSUjhpeMxf1iX5DO6XGXLFPUdD\ncytLN+2KCeWdbNsTOZLLyUilvLg/pxTnc0o0lPOyev5RnPRsCpIYCpKDixxt7GRhVS0LN+7k3epd\n7GtuBeCY3D5MOBAa/Rk7LFd3KsWRu1O1Y99HjvZWflBHm0cu5o8c3PfAkd6Ekv6MGJSji/jSrRQk\nMRQkEQ3NrSzbvIuFG3eycONOFlXtZNPOyNFGeqoxZlge5UWRv4gnlPTX3Uch2NPYwuKqnQduWliw\noZa66IX8/tnpTCgZwFmjBnHWqMEUD8wOuVpJdgkRJGY2FfgZkArc7+4/aLe+GPgdkB9t8213f8bM\nSoEVwKpo0znuflN0m1eAocC+6Lrz3X3roerojUHi7mzcUR8NjVoWVu1kxZa6A+foh+dnUV6cT3lx\n5O6iMUNzyUzX0UaiaWtz3q/Zc+Co5a33tx841ThiUA5njirgrOMLmFw2UA9PStyFHiRmlgqsBs4D\nqoF5wNXuvjymzb3AQnf/pZmNAZ5x99JokPzF3cd1sN9XgG+6e6eToTcESV1DM0uqdrFwYy2Lqnay\nsGonO/Y2AZE7qU4qzIuERlE+44t1baOncnfWbdvLq6treHV1DW+/v53GljYy0lKYVDaAs0YVcPbx\nBRxb0FdHk9JlnQ2SIG//nQiscfe10YIeAS4Blse0cSA3+joP2BxgPT3evqZWNuzY/xxDPRu2R55l\n2LC9ng/qGg60O25wX845YfCBo42Rg/vqTqokYWaMKOjLiIK+fOn0MhqaW5m7bgevrKrh1dVb+d5f\nV/C9v65geH4WZ0ZD5RPHDkyK268lcQUZJMOBqpj5amBSuza3Ay+Y2UwgBzg3Zl2ZmS0E6oDb3P31\nmHUPmFkr8DjwPU+iCz2dDQuAQX0zKBmYw+nHDWJEQQ4nFeZxUmG+7vbpRTLTUzlzVAFnjioAxlC1\no57X3qvh1VU1PLVoE7PmbiQtxZhQ0p+zji/grFEFjBmaq6MViasgT219HrjA3a+Pzk8DJrr7zJg2\n34jW8BMzOw34DTAOSAf6uvt2M5sAPAmMdfc6Mxvu7pvMrB+RIPmDu/++g/e/EbgRoLi4eMKGDZ36\nDvtuERsW67fXH3hS+lBhERnjKZuSmLGecvVXphxCU0sbCzbWRk6Draph+ZY6AAr69eGsUQV89pRC\nJo8YoFCRg0qEaySnAbe7+wXR+VsB3P37MW2WAVPdvSo6vxaY3P7i+cGui5jZdUCFu884VC1BXiNx\nd/Y1t1K3r4XdDc3UNUR+7m5oYXdDC3UNzexuaGb7niaFhYRqa13DgWsrr66uYXdDC8cN7su1k4q5\nfEKh/l+Tj0mEIEkjcrH9HGATkYvtX3D3ZTFtngUedfcHzWw08Dcip8QGATvcvdXMRgCvAycSOc2V\n7+7bzCwdmAW85O6/OlQtRxsks+ZuZPPOfTGB0ELdvmhINP4jLGLHlOpIaorRPztdYSEJY19TK08v\n2czDczawuHoXWempXFo+jGsnlzB22MfHGZPeKfSL7e7eYmYzgOeJ3Nr7W3dfZmZ3AJXu/hRwC3Cf\nmf0bkQvv17m7m9mZwB1m1gK0Aje5+w4zywGej4ZIKvAScF9QfXjwzfW8t3U3ffuk0S8znX6ZaeRm\npjMsP5N+mf3ol5kWnf6xbv987v6fWWlkpafq9IEklKyMVK6oKOKKiiKWVO/kD3M28KcFm5g1t4ry\n4nymTS7hohOH6pZw6RQ9kHgI+5pa6ZOWQorGQJJeYFd9M48tqObhORtYu20v/bPTuaKiiC9MKqZk\nYE7Y5UkIQj+1lUh6w3MkIvHi7rz1/nYeensDL674kDZ3zhxZwLTJJXzyhMEaXLIXUZDEUJCIHJ0P\ndjUwa+5GHpm3kQ/rGhmen8UXJhVzRUWRhsLvBRQkMRQkIl3T3NrGS8s/5KE5G3jr/e2kpxoXjB3C\ntMklTCzTLcTJKvSL7SKSPNJTU7jwxKFceOJQ3q/Zw8NzNvK/86v4y5ItnDmqgJ9dOZ7+ORlhlykh\n0RGJiByVfU2t/HHuRn747EoG5/bh19Mm6NbhJNPZIxINwCQiRyUrI5XpU8p45F8m09LqfPaXb/Hk\nwk1hlyUhUJCISJecUtyfp2dO4aTCfL7+6CK++/QymqNfxSy9g4JERLqsoF8fHr5+El86vZQH3lzP\nNfe/Q83uxrDLkm6iIBGRuEhPTeG/PjOWn155Mkuqd/KZu99g4cbasMuSbqAgEZG4uqy8kMdv/gRp\nqcaVv57DrLkbwy5JAqYgEZG4Gzssj6dnTGHSiAHc+qd3ufVPS2hsaQ27LAmIgkREAtE/J4MHvzSR\nr5x9LLPmVnHlr+ewZde+sMuSAChIRCQwqSnGv089gV9dewrvfbibz9z9Bu+s3R52WRJnChIRCdzU\ncUN58qunk5uZzjX3v8MDb66jNzwM3VsoSESkW4w8ph9Pzjids48fzHefXs43Zi9mX5OumyQDBYmI\ndJvczHTunTaBW84bxZOLNvHZX75F1Y76sMuSLlKQiEi3SkkxZp4zkt/+86lU19bzmZ+/wWura8Iu\nS7pAQSIiofjkCYN5asYUhuRmct0Dc/nFK2t03aSHUpCISGhKB+Xwp698gk+fNIwfPbeKrzy8gD2N\nLWGXJUdIQSIiocrOSOOuq8Zz26dH88LyD7nsnjdZW7Mn7LLkCChIRCR0Zsb1Z4zgoekT2b63iUt+\n/iYvLv8w7LKkkxQkIpIwPnHsIJ6eOYWyghxu+H0l//PiatradN0k0SlIRCShDM/PYva/nMbnJxRy\n19/eY8asBQqTBKcgEZGEk5meyo8+dxL/PvV4nnn3A3775rqwS5JDCDRIzGyqma0yszVm9u0O1heb\n2d/NbKGZLTGzi6LLS81sn5ktik6/itlmgpm9G93nXWZmQfZBRMJhZtx81rGcP+YYfvjcSt6t3hV2\nSXIQgQWJmaUC9wAXAmOAq81sTLtmtwGz3b0cuAr4Rcy69919fHS6KWb5L4EbgZHRaWpQfRCRcJkZ\nP/rcSRT07cOMWQvY3dAcdknSgSCPSCYCa9x9rbs3AY8Al7Rr40Bu9HUesPlQOzSzoUCuu7/tkSeX\nfg9cGt+yRSSR5GdncOdV5VTtqOc7Ty7VQ4sJKMggGQ5UxcxXR5fFuh241syqgWeAmTHryqKnvF41\nszNi9ll9mH0CYGY3mlmlmVXW1Gj4BZGebGLZAL5+7iieXLSZxxdsCrscaSfIIOno2kX7PyWuBh50\n90LgIuAhM0sBtgDF0VNe3wD+aGa5ndxnZKH7ve5e4e4VBQUFR90JEUkMX/3kcUweMYDvPLmU9/XA\nYkIJMkiqgaKY+UI+fupqOjAbwN3fBjKBQe7e6O7bo8vnA+8Do6L7LDzMPkUkCaWmGHdeWU5megoz\n/7iQhmYNQZ8oggySecBIMyszswwiF9OfatdmI3AOgJmNJhIkNWZWEL1Yj5mNIHJRfa27bwF2m9nk\n6N1aXwT+HGAfRCSBDMnL5MefP5nlW+r4wbMrwy5HogILEndvAWYAzwMriNydtczM7jCzi6PNbgFu\nMLPFwCzguuhF9DOBJdHljwE3ufuO6DY3A/cDa4gcqTwbVB9EJPGcM/oYvnx6GQ++tV7DqCQI6w13\nQFRUVHhlZWXYZYhInDS2tHL5L95i0859PPu1MxialxV2SUnJzOa7e8Xh2unJdhHpcfqkpXL31eU0\ntbTxtVmLaGltC7ukXk1BIiI90oiCvnzv0nHMXb+Du19eE3Y5vZqCRER6rMtPKeTy8uHc/fJ7zFm7\nPexyei0FiYj0aHdcOo6SgTl8/ZFF1O5tCrucXklBIiI9Wt8+adx9dTnb9zbyrccWawiVEChIRKTH\nGzc8j1svHM1LK7by4Fvrwy6n11GQiEhS+NLppZxzwmC+/8xKlm7SkPPdSUEiIknBzPh/nz+Z/jnp\nzJy1kL2NLWGX1GsoSEQkaQzIyeDOK8tZv30v/+fPy8Iup9dQkIhIUjnt2IHM/NRIHl9QzRMLqw+/\ngXSZgkREks6/fuo4JpYO4LYnlrJu296wy0l6ChIRSTppqSncedV40lJTmDlrAY0tGnI+SAoSEUlK\nw/Kz+NHnTmLppjp+9NyqsMtJagoSEUlaF4wdwhdPK+E3b6zj5ZUacj4oChIRSWr/edFoThjSj1tm\nL2b7nsawy0lKChIRSWqZ6ancdXU5u/Y1a5TggChIRCTpjTqmH1eeWsTD72xg4/b6sMtJOgoSEekV\nvn7uKFJTjB+/oAvv8aYgEZFe4ZjcTKZPKeOpxZt5t1pjccWTgkREeo1/OetY+men88PnVoZdSlJR\nkIhIr5Gbmc6MT43kjTXbeG11TdjlJA0FiYj0KtdOLqawfxY/eHYlbW36Eqx4UJCISK/SJy2Vb55/\nPMu31PHU4s1hl5MUAg0SM5tqZqvMbI2ZfbuD9cVm9nczW2hmS8zsog7W7zGzb8YsW29m75rZIjOr\nDLJ+EUlOF588jDFDc/nxC6s0DlccBBYkZpYK3ANcCIwBrjazMe2a3QbMdvdy4CrgF+3W/xR4toPd\nf9Ldx7t7RZzLFpFeICXF+PaFJ1Bdu48/zNkYdjk9XpBHJBOBNe6+1t2bgEeAS9q1cSA3+joPOHCc\naWaXAmsBfTuNiMTdmaMKmHLcIH7+8nvUNTSHXU6PFmSQDAeqYuaro8ti3Q5ca2bVwDPATAAzywH+\nA/huB/t14AUzm29mN8a7aBHpPf5j6gnU1jfz61ffD7uUHi3IILEOlrW/ReJq4EF3LwQuAh4ysxQi\nAfJTd9/TwT5Od/dTiJwy+6qZndnhm5vdaGaVZlZZU6Pb/ETk404szOPik4fxmzfW8WFdQ9jl9FhB\nBkk1UBQzX0jMqauo6cBsAHd/G8gEBgGTgB+Z2Xrg68B/mtmMaLvN0Z9bgSeInEL7GHe/190r3L2i\noKAgXn0SkSTzzfOPp7XNufOl1WGX0mMFGSTzgJFmVmZmGUQupj/Vrs1G4BwAMxtNJEhq3P0Mdy91\n91LgTuC/3f3nZpZjZv2i7XOA84GlAfZBRJJc8cBsrplUwqPzqliztaOTIHI4gQWJu7cAM4DngRVE\n7s5aZmZ3mNnF0Wa3ADeY2WJgFnCdux/qCaFjgDei7ecCf3X354Lqg4j0DjM/dRzZGWn8SEOnHBU7\n9L/byaGiosIrK/XIiYgc3N1/e4+fvLiax28+jQklA8IuJyGY2fzOPGahJ9tFRIDpZ5RR0K8P339m\nJb3hD+x4UpCIiADZGWl8/dyRVG6o5aUVW8Mup0dRkIiIRF1ZUcSIQTn88LmVtLS2hV1Oj6EgERGJ\nSktN4d+nHs+arXt4fEF12OX0GAoSEZEYF4wdQnlxPv/z4mr2NWlAx87odJCY2RQz+1L0dYGZlQVX\nlohIOMyMWy8czYd1jTzw1rqwy+kROhUkZvZfRMa+ujW6KB34Q1BFiYiEaWLZAM4dPZhfvvI+tXub\nwi4n4XX2iOQy4GJgLxwYpqRfUEWJiITt36eewN7GFu75+5qwS0l4nQ2SpugT5w4HhicREUlao47p\nx+cmFPL7tzdQtaM+7HISWmeDZLaZ/RrIN7MbgJeA+4IrS0QkfP923ijM4KcvakDHQ+lUkLj7j4HH\ngMeB44H/4+53B1mYiEjYhuZl8aXTy3hi0SaWb64Lu5yEddggMbNUM3vJ3V9092+5+zfd/cXuKE5E\nJGw3n3UsuZnp/FADOh7UYYPE3VuBejPL64Z6REQSSl52OjM+eRyvrq7hrTXbwi4nIXX2GkkD8K6Z\n/cbM7to/BVmYiEiimHZaCcPzs/j+sytpa9OAju11Nkj+CnwHeA2YHzOJiCS9zPRUvnHeKN7dtIu/\nvrsl7HISTlpnGrn776LfcjgqumiVuzcHV5aISGK5tHw4972+lh+/sIoLxg4hI00jTO3X2Sfbzwbe\nA+4BfgGsNrMzA6xLRCShpKYY/3HhCWzYXs+jlVVhl5NQOhupPwHOd/ez3P1M4ALgp8GVJSKSeM4e\nVUB5cT73v76WVl0rOaCzQZLu7qv2z7j7aiLjbYmI9BpmxvVTRrBhez0vrfgw7HISRmeDpDJ6x9bZ\n0ek+dLFdRHqhC8YeQ2H/LO5/fW3YpSSMzgbJzcAy4F+BrwHLgZuCKkpEJFGlpabw5dPLmLe+lkVV\nO8MuJyF0NkjSgJ+5++XufhlwF5AaXFkiIonrilOL6JeZpqOSqM4Gyd+ArJj5LCIDN4qI9Dp9+6Tx\nhYnFPLv0A6prNTJwZ4Mk09337J+Jvs4OpiQRkcR33emlGPDAm+vDLiV0nQ2SvWZ2yv4ZM6sA9h1u\nIzObamarzGyNmX27g/XFZvZ3M1toZkvM7KIO1u8xs292dp8iIt1haF4Wnz5pKI/Oq6KuoXc/n93Z\nIPka8L9m9rqZvQY8Asw41AZmlkrkAcYLgTHA1WY2pl2z24DZ7l4OXEXkYcdYPwWePcJ9ioh0ixvO\nGMGexhYendu7H1DsbJCUAeVE7t56EVhF9NsSD2EisMbd17p7E5HwuaRdGwdyo6/zgM37V5jZpcBa\nIneLHck+RUS6xbjheUweMYAH3lxHS2tb2OWEprNB8h13rwPygfOAe4FfHmab4UBsTFdHl8W6HbjW\nzKqBZ4CZcOCrfP8D+O5R7JPoPm40s0ozq6ypqTlMqSIiR+f6KSPYvKuBZ5Z+EHYpoelskLRGf34a\n+JW7/xnIOMw21sGy9kcxVwMPunshcBHwkJmlEAmQn8Ze4D+CfUYWut/r7hXuXlFQUHCYUkVEjs6n\nThjMiIIc7n99Le69c9iUzgbJpuh3tl8BPGNmfTqxbTVQFDNfSMypq6jpwGwAd38byAQGAZOAH5nZ\neuDrwH+a2YxO7lNEpNukpBjTp5SxpHoX89bXhl1OKDobJFcAzwNT3X0nMAD41mG2mQeMNLOy6BD0\nVwFPtWuzETgHwMxGEwmSGnc/w91L3b0UuBP4b3f/eSf3KSLSrS4vL6R/djr39dIHFDsVJO5e7+5/\ncvf3ovNb3P2Fw2zTQuTOrueBFUTuzlpmZneY2cXRZrcAN5jZYmAWcJ0f4tjwYPvsTB9ERIKSlZHK\ntMklvLTiQ9Zt2xt2Od3OesM5vYqKCq+srAy7DBFJYjW7Gzn9By9z5alF/N9Lx4VdTlyY2Xx3rzhc\nO33Fl4hIHBT068Ol5cP43/lV1O5tCrucbqUgERGJk+vPGEFDcxsPv7Mh7FK6lYJERCRORh3Tj7NG\nFfC7tzfQ2NJ6+A2ShIJERCSOrj+jjJrdjTy1qPc8maAgERGJoynHDeKEIf34zRvres0DigoSEZE4\nMjOuP2MEKz/YzRtrtoVdTrdQkIiIxNlnTh5KQb8+3Pf6urBL6RYKEhGROOuTlsp1nyjltdU1rPpg\nd9jlBE5BIiISgC9MLCYzPYXfvJH8w6YoSEREAtA/J4PPTyjiyYWb2bq7IexyAqUgEREJyJenlNHc\n1sZDbyf3A4oKEhGRgJQNyuHc0cfwhzkb2NeUvA8oKkhERAJ0wxkjqK1v5vEF1WGXEhgFiYhIgE4t\n7c/JhXn89o11tLUl5wOKChIRkQCZGdPPGMHabXt5eeXWsMsJhIJERCRgF40bwvD8rKT9BkUFiYhI\nwNJSU/jS6aW8s24H71bvCrucuFOQiIh0gytOLaJvnzTuT8IHFBUkIiLdIDcznatOLeIvS7aweee+\nsMuJKwWJiEg3ue70UgB+99b6UOuINwWJiEg3KeyfzYXjhvDHuRvZ09gSdjlxoyAREelGN5wxgt0N\nLTw6ryrsUuJGQSIi0o1OLspnYukAHnhzHS2tbWGXExcKEhGRbjb9jDKqa/fx/LIPwy4lLgINEjOb\namarzGyNmX27g/XFZvZ3M1toZkvM7KLo8olmtig6LTazy2K2WW9m70bXVQZZv4hIEM4dfQylA7O5\n7/W1SfG97oEFiZmlAvcAFwJjgKvNbEy7ZrcBs929HLgK+EV0+VKgwt3HA1OBX5tZWsx2n3T38e5e\nEVT9IiJBSU0xpk8pY1HVThZsrA27nC4L8ohkIrDG3de6exPwCHBJuzYO5EZf5wGbAdy93t3339KQ\nGW0nIpI0PjuhkLysdO57red/r3uQQTIciL0toTq6LNbtwLVmVg08A8zcv8LMJpnZMuBd4KaYYHHg\nBTObb2Y3HuzNzexGM6s0s8qampqu90ZEJI6yM9K4dnIxzy//gA3b94ZdTpcEGSTWwbL2RxZXAw+6\neyFwEfCQmaUAuPs77j4WOBW41cwyo9uc7u6nEDll9lUzO7OjN3f3e929wt0rCgoK4tEfEZG4+uJp\npaSlGA+8uT7sUrokyCCpBopi5guJnrqKMR2YDeDubxM5jTUotoG7rwD2AuOi8/tPf20FniByCk1E\npMc5JjeTi08ezuzKKnbVN4ddzlELMkjmASPNrMzMMohcTH+qXZuNwDkAZjaaSJDURLdJiy4vAY4H\n1ptZjpn1iy7PAc4ncmFeRKRHuv6MMuqbWnl4bs/9XvfAgiR6TWMG8DywgsjdWcvM7A4zuzja7Bbg\nBjNbDMwCrvPIvXBTgMVmtojIUcdX3H0bcAzwRrT9XOCv7v5cUH0QEQna6KG5TDluEL97az1NLT3z\nAUVLhnuYD6eiosIrK/XIiYgkpldWbeW6B+bxP1eczOWnFIZdzgFmNr8zj1noyXYRkZCdNaqAkYP7\ncv/r63rkA4oKEhGRkJkZ159RxvItdbz9/vawyzliChIRkQRwyfjhDOqb0SO/111BIiKSADLTU/ni\naaX8fVUNa7buDrucI6IgERFJENdMKqZPWgq/eaNnDZuiIBERSRAD+/bhsvLhPLFwE7v29ZwHFBUk\nIiIJZNppJTQ0t/H4/OqwS+k0BYmISAIZOyyPU4rz+cOcDT3mVmAFiYhIgpl2Wglrt+3lrR5yK7CC\nREQkwVw4bigDcjJ46O2eMf6WgkREJMFkpqdyRUURL674kC279oVdzmEpSEREEtA1k4ppc2fW3KrD\nNw6ZgkREJAEVDcjmk8cPZtb/zmv1AAAKPUlEQVTcjTS3JvaowAoSEZEENW1yCTW7G3lh2Ydhl3JI\nChIRkQR15qgCigZk8dCc9WGXckgKEhGRBJWaYlwzqYQ5a3fw3oeJO/6WgkREJIFdUVFERloKD81J\n3FuBFSQiIglsQE4G/3TiUP60YBN7GlvCLqdDChIRkQR37Wkl7Gls4cmFm8IupUMKEhGRBFdelM/Y\nYbkJO/6WgkREJMGZGdMml7Dyg91UbqgNu5yPUZCIiPQAl4wfTr/MtIQcf0tBIiLSA2RlpPL5CUU8\nu3QLNbsbwy7nIwINEjObamarzGyNmX27g/XFZvZ3M1toZkvM7KLo8olmtig6LTazyzq7TxGRZHXN\n5GKaW53ZlYk1/lZgQWJmqcA9wIXAGOBqMxvTrtltwGx3LweuAn4RXb4UqHD38cBU4NdmltbJfYqI\nJKVjC/oy5bhBPDxnA61tiXPRPcgjkonAGndf6+5NwCPAJe3aOJAbfZ0HbAZw93p333/DdGa0XWf3\nKSKStK6dXMLmXQ28vHJr2KUcEGSQDAdij7+qo8ti3Q5ca2bVwDPAzP0rzGySmS0D3gVuigZLZ/a5\nf/sbzazSzCpramq62hcRkYRw7ujBDMnNTKgn3YMMEutgWftjsauBB929ELgIeMjMUgDc/R13Hwuc\nCtxqZpmd3CfR7e919wp3rygoKDjqToiIJJK01BS+MKmY11bXsH7b3rDLAYINkmqgKGa+kOipqxjT\ngdkA7v42kdNYg2IbuPsKYC8wrpP7FBFJaledWkRaivHwO4lxVBJkkMwDRppZmZllELmY/lS7NhuB\ncwDMbDSRIKmJbpMWXV4CHA+s7+Q+RUSS2uDcTC4YN4TZldXsa2oNu5zggiR6TWMG8DywgsjdWcvM\n7A4zuzja7BbgBjNbDMwCrvPI8/9TgMVmtgh4AviKu2872D6D6oOISKKaNrmEXfuaeXpJ+CdlLBHH\nbYm3iooKr6ysDLsMEZG4cXfO/+lrZGWk8tSMKYG8h5nNd/eKw7XTk+0iIj2QmTHttBKWVO9icdXO\nUGtRkIiI9FCXlQ8nOyM19FuBFSQiIj1Uv8x0LisfztOLN1O7tym0OhQkIiI92LWTS2hsaeOx+dWh\n1aAgERHpwUYPzeXU0v784Z0NtIU0/paCRESkh7t2cgkbttfz+pptoby/gkREpIebOm4Ig/pmhPal\nVwoSEZEerk9aKledWszLKz+kura+299fQSIikgSunlQMwKy5G7v9vRUkIiJJYHh+FueMPoZH51XR\n2NK9428pSEREksS0ySVs29PEc0s/6Nb3VZCIiCSJKccNonRgdrdfdFeQiIgkiZQU49rJJVRuqGX5\n5rrue99ueycREQnc5yYU0icthT9045deKUhERJJIfnYGF588jCcXbqKuoblb3lNBIiKSZKadVkJ9\nUytPLNjULe+nIBERSTInFeZzcmEeD83ZQHd8eaGCREQkCV07uQSAmj2Ngb9XWuDvICIi3e7yUwr5\n3IRCzCzw91KQiIgkodSU4ANkP53aEhGRLlGQiIhIlyhIRESkSxQkIiLSJYEGiZlNNbNVZrbGzL7d\nwfpiM/u7mS00syVmdlF0+XlmNt/M3o3+/FTMNq9E97koOg0Osg8iInJogd21ZWapwD3AeUA1MM/M\nnnL35THNbgNmu/svzWwM8AxQCmwDPuPum81sHPA8MDxmu2vcvTKo2kVEpPOCPCKZCKxx97Xu3gQ8\nAlzSro0DudHXecBmAHdf6O6bo8uXAZlm1ifAWkVE5CgF+RzJcKAqZr4amNSuze3AC2Y2E8gBzu1g\nP58FFrp77OOZD5hZK/A48D3vYAwAM7sRuDE6u8fMPgB2xTTJO8R87OtBRI6Quqr9+3WlbUfrO7Os\np/b5YOvU546Xqc8d9zle/T1YTUfTLl59DuozLulUK3cPZAI+D9wfMz8NuLtdm28At0RfnwYsB1Ji\n1o8F3geOjVk2PPqzH/AC8MVO1nNvZ+fbva6M03+Pe+PVtqP1nVnWU/t8sHXqs/p8JH2OV3+PpM9H\n87t8NH0O+jM+3BTkqa1qoChmvpDoqasY04HZAO7+NpBJJEExs0LgCSJB8f7+Ddx9U/TnbuCPRE6h\ndcbTRzDffl08HMk+D9e2o/WdWdZT+3ywdepzx8vU58Tp89H8Lh9seWf7GER/D8miqRX/HZulAauB\nc4BNwDzgC+6+LKbNs8Cj7v6gmY0G/kbklFge8Cpwh7s/3m6f+e6+zczSgVnAS+7+q0A6EXnPSnev\nCGr/iUh97h16W597W3+h+/oc2BGJu7cAM4jccbWCyN1Zy8zsDjO7ONrsFuAGM1tMJBSu80iyzQCO\nA77T7jbfPsDzZrYEWEQkoO4Lqg9R9wa8/0SkPvcOva3Pva2/0E19DuyIREREegc92S4iIl2iIBER\nkS5RkIiISJcoSLrAzC41s/vM7M9mdn7Y9XQHMxthZr8xs8fCriUoZpZjZr+LfrbXhF1Pd+gNn2t7\nvfT3d7SZ/crMHjOzm+O1314bJGb2WzPbamZL2y0/5ECTsdz9SXe/AbgOuDLAcuMiTn1e6+7Tg600\n/o6w75cDj0U/24s/trMe4kj63FM/1/aOsM896vf3YI6wzyvc/SbgCiB+twV3x1OPiTgBZwKnAEtj\nlqUSeZJ+BJABLAbGACcCf2k3DY7Z7ifAKWH3qZv7/FjY/Qmw77cC46Nt/hh27d3R5576ucapzz3i\n9zdefSbyx9FbRJ7ri0sNvfY72939NTMrbbf4wECTAGb2CHCJu38f+Kf2+zAzA34APOvuC4KtuOvi\n0eee6kj6TmRUhkIizyr12KP2I+zzcpLAkfTZzFbQg35/D+ZIP2d3fwp4ysz+SmR0kC7rsb8kAelo\noMnhB2kLMJPIQJOfM7ObgiwsQEfUZzMbaGa/AsrN7NagiwvYwfr+J+CzZvZLQhhuImAd9jnJPtf2\nDvY5J8Pv78Ec7HM+28zuMrNfE/najrjotUckB2EdLDvoE5vufhdwV3DldIsj7fN2IFl+6Trsu7vv\nBb7U3cV0k4P1OZk+1/YO1udk+P09mIP1+RXglXi/mY5IPqozA00mm97Y5/16Y9/VZ/U57hQkHzUP\nGGlmZWaWAVwFPBVyTUHrjX3erzf2XX1Wn+Ou1waJmc0C3gaON7NqM5vuBxloMsw646k39nm/3th3\n9Vl9ppv6rEEbRUSkS3rtEYmIiMSHgkRERLpEQSIiIl2iIBERkS5RkIiISJcoSEREpEsUJCIi0iUK\nEhER6RIN2igSAjMbC/wMKAYeAgYDv3f3eaEWJnIU9GS7SDczs0xgAfB5YC2wEpjv7peHWpjIUdIR\niUj3OxdYuH/so+igej8JtySRo6drJCLdr5zIEQlmNgzY4+5vhluSyNFTkIh0v0Yi3w8B8H0i36kt\n0mMpSES63x+BM81sFbAYeNvM7gy5JpGjpovtIiLSJToiERGRLlGQiIhIlyhIRESkSxQkIiLSJQoS\nERHpEgWJiIh0iYJERES6REEiIiJd8v8BWwZf85kISgUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1bce2a184e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def test_ridge_alpha(*data):\n",
    "    X_train, X_test, y_train, y_test,train,test = data\n",
    "    alphas = [0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 50, 100, 200, 500, 1000]\n",
    "    scores = []\n",
    "    for i, alpha in enumerate(alphas):\n",
    "        ridgeRegression = linear_model.Ridge(alpha=alpha)\n",
    "        ridgeRegression.fit(X_train, y_train)\n",
    "        scores.append(ridgeRegression.score(X_test, y_test))\n",
    "    return alphas, scores\n",
    "\n",
    "def show_plot(alphas, scores):\n",
    "    figure = plt.figure()\n",
    "    ax = figure.add_subplot(1, 1, 1)\n",
    "    ax.plot(alphas, scores)\n",
    "    ax.set_xlabel(r\"$\\alpha$\")\n",
    "    ax.set_ylabel(r\"score\")\n",
    "    ax.set_xscale(\"log\")\n",
    "    ax.set_title(\"Ridge\")\n",
    "    plt.show()\n",
    "    \n",
    "alphas, scores = test_ridge_alpha(X_train, X_test, y_train, y_test,train,test)\n",
    "show_plot(alphas, scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 用Lasso回归模型对房价进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "权重向量:[  0.00000000e+00   1.87196575e-06   0.00000000e+00  -0.00000000e+00\n",
      "  -0.00000000e+00  -0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   2.01665837e-03   1.14122359e-03   2.70696968e-05\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   1.10230927e-04   0.00000000e+00\n",
      "  -0.00000000e+00  -0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00  -0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00  -0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "  -0.00000000e+00  -0.00000000e+00   0.00000000e+00  -0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   8.03040919e-05   0.00000000e+00  -0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00  -0.00000000e+00\n",
      "  -2.59192632e-05   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   1.71770747e-04\n",
      "   0.00000000e+00   0.00000000e+00   2.27419066e-04   0.00000000e+00\n",
      "   3.87009184e-05   0.00000000e+00   0.00000000e+00  -2.76868581e-08\n",
      "  -4.40034882e-12   0.00000000e+00   1.70492658e-07  -3.34230080e-11\n",
      "   0.00000000e+00   2.50809879e-08   6.55234640e-12   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   4.10030858e-07  -2.59584712e-10   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   9.52083614e-08\n",
      "   1.99834473e-11   0.00000000e+00  -9.68747286e-09  -1.24214738e-11\n",
      "   0.00000000e+00], b的值为:4.78\n",
      "损失函数的值:0.03\n",
      "预测性能得分: 0.80\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\coordinate_descent.py:491: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.\n",
      "  ConvergenceWarning)\n"
     ]
    }
   ],
   "source": [
    "def test_lasso(*data):\n",
    "    X_train, X_test, y_train, y_test,train,test = data\n",
    "    lassoRegression = linear_model.Lasso()\n",
    "    lassoRegression.fit(X_train, y_train)\n",
    "    print(\"权重向量:%s, b的值为:%.2f\" % (lassoRegression.coef_, lassoRegression.intercept_))\n",
    "    print(\"损失函数的值:%.2f\" % np.mean((lassoRegression.predict(X_test) - y_test) ** 2))\n",
    "    print(\"预测性能得分: %.2f\" % lassoRegression.score(X_test, y_test))\n",
    "    ##生成预测\n",
    "    submission = pd.DataFrame()\n",
    "    submission['Id'] = test.Id\n",
    "    #根据上面所做的模型，从测试数据中选择特性\n",
    "    feats = test.select_dtypes(\n",
    "            include=[np.number]).drop(['Id'], axis=1).interpolate()\n",
    "    #生成预测\n",
    "    predictions = lassoRegression.predict(feats)\n",
    "    #预测转换成正确的形式，用np.exp()来做预测，因为之前已经取了对数。\n",
    "    final_predictions = np.exp(predictions)\n",
    "    submission['SalePrice'] = final_predictions\n",
    "    #创建预测\n",
    "    submission.to_csv('LassoRegressionPredictions.csv', index=False)\n",
    "    \n",
    "test_lasso(X_train, X_test, y_train, y_test,train,test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\coordinate_descent.py:491: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.\n",
      "  ConvergenceWarning)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEaCAYAAADg2nttAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VOW9x/HPb7IQwk4IS9h3RSRR\nEXfccamEutTCbWtre2t7W6z1Xm3trW2tbW+1176s9tqFLtdqb6UWqYKiuG8ttoCyCsgiyBIk7BDI\n/rt/zARDSJgJ5ORMZr7v12temTnnOZNfeF7Jl+ecOc9j7o6IiMjRRMIuQEREkp/CQkRE4lJYiIhI\nXAoLERGJS2EhIiJxKSxERCQuhYXIcTKzX5nZd46y381sWGvWJNLSTPdZiMRnZuuBXkANsB94Dpjq\n7vsTONaB4e6+JtAiRQKkkYVI4ia6e0egCDgF+FbI9Yi0GoWFSDO5+1ZgLtHQwMweNrMf1u03s9vN\nrMTMtpjZ5+sfa2Z5ZjbbzPaa2Xwz+6GZvVlv/wlm9oKZ7TSzVWZ2fWv9XCJHo7AQaSYz6wdcARxx\nWsnMLgduAy4FhgOXNGjyEFAG9AY+G3vUHdsBeAH4E9ATmAL8wsxOavmfQqR5FBYiiXvSzPYBG4Ft\nwPcaaXM98L/uvszdy4C76naYWQZwLfA9dz/g7u8Cf6h37FXAenf/X3evdve3gSeA64L5cUQSp7AQ\nSdzH3b0TcAFwAtCjkTYFRMOkzoZ6z/OBzAb76z8fCJxhZrvrHsCniI5CREKlsBBpJnd/DXgYuK+R\n3SVA/3qvB9R7XgpUA/3qbavfdiPwmrt3rffo6O7/1jKVixw7hYXIsfkZcKmZFTXY/jjwOTMbZWa5\n1DtV5e41wEzgLjPLNbMTgBvqHfs0MMLMPmNmWbHH6WZ2YsA/i0hcCguRY+DupcAjwHcabH+WaJC8\nTPQC+MsNDp0KdAG2Ao8CjwEVsWP3AROAycCWWJt7gXZB/RwiidJNeSIhMrN7gd7u/tm4jUVCpJGF\nSCuK3UcxxqLGAV8A/hp2XSLxZIZdgEia6UT01FMB0Y/f/hR4KtSKRBKg01AiIhKXTkOJiEhcCgsR\nEYkrZa5Z9OjRwwcNGhR2GSIibcrChQu3u3t+vHaBhkVsUrUHgAzgt+5+T4P9A4jOjdM11uYOd58T\n2/ctop8UqQG+5u5zj/a9Bg0axIIFC1r+hxARSWFmtiF+qwDDIjZp2kNEZ9/cBMw3s1mxydPq3Ak8\n7u6/NLNRwBxgUOz5ZOAkop8aedHMRsTugBURkVYW5DWLccAad1/n7pXAdGBSgzYOdI4970L0rlVi\n7aa7e4W7v0/0TthxAdYqIiJHEWRY9OXwGTU3xbbVdxfwaTPbRHRUcXMzjhURkVYSZFhYI9sa3tQx\nBXjY3fsBVwKPmlkkwWMxs5vMbIGZLSgtLT3ugkVEpHFBhsUmDp9+uR8fnWaq8wWis3Ti7vOAHKJr\nBCRyLO4+zd3HuvvY/Py4F/NFROQYBRkW84HhZjbYzLKJXrCe1aDNB8DFALFpmHOIzvk/C5hsZu3M\nbDDR5Sn/GWCtIiJyFIGFhbtXE52OeS6wguinnpab2d1mVhxr9h/AF81sMdH5cj7nUcuJjjjeBZ4D\nvhrUJ6Eqq2t5acWHlOw5GMTbi4ikhJSZG2rs2LF+LPdZbN59kHPueZnbJoxg6kXDA6hMRCR5mdlC\ndx8br13aT/fRt2t7xg3qzlOLtpAqwSki0tLSPiwAJhYVsHrbflZu3Rd2KSIiSUlhAVw5ujcZEWPW\n4iM+cCUiIigsAMjr2I5zh/Vg9mKdihIRaYzCImZSUQGbdh3k7Q92h12KiEjSUVjETDipN+0yI8xa\ntDnsUkREko7CIqZju0wuPrEnzywtobqmNuxyRESSisKinuLCArbvr2Teuh1hlyIiklQUFvVcMLIn\nndplMmuRPhUlIlKfwqKenKwMLhvdm+eWbaW8SussiYjUUVg0UFxYwL6Kal5dpSnPRUTqKCwaOHto\nHj06ZjNbN+iJiByisGggMyPCx07uw4srPmR/RXXY5YiIJAWFRSOKiwqoqK7l+eVbwy5FRCQpKCwa\nceqAbvTt2l5zRYmIxCgsGmFmTCws4I3V29mxvyLsckREQqewaMKkogJqap05y3QqSkREYdGEE3p3\nYnjPjszWDXoiIgqLppgZxYUF/HP9Trbs1vrcIpLeFBZHMbGwAICnl2h0ISLpTWFxFIN6dKCwf1ee\n0qkoEUlzCos4igsLWL5lL2tL94ddiohIaBQWcVw1pg9maCZaEUlrCos4enXO4czBeVqfW0TSmsIi\nAZOKCli3vYxlm/eGXYqISCgUFgm4YnQfsjKMWYu1PreIpCeFRQK65GZx/oh8nl5SQm2tTkWJSPpR\nWCRoYmEBJXvKmb9+Z9iliIi0OoVFgi4d1Yv2WRk8pZloRSQNKSwSlJudyaWjevHs0hKqamrDLkdE\npFUpLJqhuLCAXQeqeHP19rBLERFpVQqLZhg/Ip8u7bO0KJKIpB2FRTNkZ0a48uTezF2+lYOVNWGX\nIyLSahQWzTSxsIADlTW8tPLDsEsREWk1CotmOmNwHj07tdNcUSKSVhQWzZQRMa4aU8Crq0rZc7Aq\n7HJERFqFwuIYTCoqoLKmlrlan1tE0oTC4hiM6deFgXm5+lSUiKQNhcUxqFuf++9rt7NtX3nY5YiI\nBC7QsDCzy81slZmtMbM7Gtl/v5ktij3eM7Pd9fb9xMyWm9kKM3vQzCzIWpuruLCAWoc5S0rCLkVE\nJHCBhYWZZQAPAVcAo4ApZjaqfht3v9Xdi9y9CPg5MDN27NnAOcAYYDRwOnB+ULUei+G9OnFin86a\nK0pE0kKQI4txwBp3X+fulcB0YNJR2k8BHos9dyAHyAbaAVlA0t3YUFxYwDsf7GbjzgNhlyIiEqgg\nw6IvsLHe602xbUcws4HAYOBlAHefB7wClMQec919RSPH3WRmC8xsQWlpaQuXH9/Ewj4AutAtIikv\nyLBo7BpDUysHTQZmuHsNgJkNA04E+hENmIvMbPwRb+Y+zd3HuvvY/Pz8Fio7cf265XLawG7MVliI\nSIoLMiw2Af3rve4HNPVXdTIfnYICuBp4y933u/t+4FngzECqPE6TigpYuXUfq7buC7sUEZHABBkW\n84HhZjbYzLKJBsKsho3MbCTQDZhXb/MHwPlmlmlmWUQvbh9xGioZXHlyHzIiWp9bRFJbYGHh7tXA\nVGAu0T/0j7v7cjO728yK6zWdAkx39/qnqGYAa4GlwGJgsbvPDqrW49GjYzvOHprH7MUlHP4jiIik\njswg39zd5wBzGmz7boPXdzVyXA3wpSBra0nFhQXcPmMJizbu5pQB3cIuR0SkxekO7hZw2ejeZGdG\neEoz0YpIilJYtIDOOVlcNLInzywtoaZWp6JEJPUoLFpIcVEBpfsqeGvdjrBLERFpcQqLFnLRCT3p\n2C5TiyKJSEpSWLSQnKwMJpzUiznLSqio1vrcIpJaFBYtqLiwgH3l1by2qvWnHhERCZLCogWdM6wH\n3Ttka64oEUk5CosWlJUR4cqTe/Piig8pq6gOuxwRkRajsGhhk4r6Ul5VywvvJt2M6iIix0xh0cJO\nG9CNgi45OhUlIilFYdHCIhFjYmEBr79Xyq6yyrDLERFpEQqLAEwsLKC61nl22dawSxERaREKiwCc\nVNCZofkdeGqRpi0XkdSgsAiAmVFc2Jd/rt/J1j3lYZcjInLcFBYBKS4qwB2eXqIL3SLS9iksAjK4\nRwdO7ttF05aLSEpQWARoUlEBSzfv4f3tZWGXIiJyXBQWAbpqTAFmaCZaEWnzFBYB6t0lh3GDuvPU\n4s1an1tE2jSFRcCKiwpYV1rG8i17wy5FROSYKSwCduXoPmRGjNma/kNE2jCFRcC6dchm/Ih8Zi/e\nQq3W5xaRNkph0QqKCwvYsqechR/sCrsUEZFjorBoBZeO6kVOVkTTf4hIm6WwaAUd2mVyyYm9mLN0\nK1U1tWGXIyLSbAqLVlJcWMDOskr+tmZ72KWIiDSbwqKVnD8yn845mVoUSUTaJIVFK2mXmcHlo3sz\nd9lWyqtqwi5HRKRZFBataFJRX8oqa3h55bawSxERaRaFRSs6c0ge+Z3aaa4oEWlzFBatKCNifOzk\nPry8aht7y6vCLkdEJGEKi1ZWXFRAZXUtc7U+t4i0IQqLVnZK/670795en4oSkTZFYdHKoutzF/D3\ntTvYvr8i7HJERBKisAhBcWFfamqdOUtLwi5FRCQhCosQjOzdiZG9Oml9bhFpMxQWISkuKmDhhl1s\n2nUg7FJEROJSWISkuLAAgNmLdSpKRJJfoGFhZpeb2SozW2NmdzSy/34zWxR7vGdmu+vtG2Bmz5vZ\nCjN718wGBVlra+vfPZdTBnTlyXc2a1EkEUl6CYeFmZ1rZjfGnueb2eA47TOAh4ArgFHAFDMbVb+N\nu9/q7kXuXgT8HJhZb/cjwH+7+4nAOCDl5sj49BkDWfXhPqbP3xh2KSIiR5VQWJjZ94BvAt+KbcoC\n/hjnsHHAGndf5+6VwHRg0lHaTwEei32/UUCmu78A4O773T3lTu5fc2pfzh6ax4/nrKBkz8GwyxER\naVKiI4urgWKgDMDdtwCd4hzTF6j/X+ZNsW1HMLOBwGDg5dimEcBuM5tpZu+Y2X/HRioNj7vJzBaY\n2YLS0tIEf5TkYWbcc80YqmprufOvy3DX6SgRSU6JhkWlR/+SOYCZdUjgGGtkW1N/DScDM9y9bu7u\nTOA84DbgdGAI8Lkj3sx9mruPdfex+fn5CZSUfAbk5XLbhJG8tHIbs5foYreIJKdEw+JxM/s10NXM\nvgi8CPwmzjGbgP71XvcDmrqxYDKxU1D1jn0ndgqrGngSODXBWtucG88ZTGH/rtw1azk7yyrDLkdE\n5AgJhYW73wfMAJ4ARgLfdfefxzlsPjDczAabWTbRQJjVsJGZjQS6AfMaHNvNzOqGCxcB7yZSa1uU\nETF+cu0Y9pVXcffs5WGXIyJyhLhhYWYZZvaiu7/g7re7+211F56PJjYimArMBVYAj7v7cjO728yK\n6zWdAkz3eifsY6ejbgNeMrOlRE9pxRvJtGkje3fiqxcO48lFW3h55YdhlyMichhL5KKqmc0CPuPu\ne4Iv6diMHTvWFyxYEHYZx6WyupaJP3+TveVVPH/reDrlZIVdkoikODNb6O5j47VL9JpFObDUzH5n\nZg/WPY6vRGkoOzPCvdeN4cO95dz73MqwyxEROSQzwXbPxB4SsKL+Xfn8OYP57ZvvM3FMAWcMyQu7\nJBGRhC9w/4Hop5UWxh5/im2TAPz7hBEM6J7LHTOXUl5VE/8AEZGAJXoH9wXAaqLTd/wCeM/MxgdY\nV1rLzc7knmtO5v3tZfzsxdVhlyMikvA1i58CE9z9fHcfD1wG3B9cWXL2sB5MPr0/v3ljHUs3Je3n\nCkQkTSQaFlnuvqruhbu/R3R+KAnQt648kbwO2XzjiSVU1dSGXY6IpLFEw2JB7JNQF8QevyF67UIC\n1KV9Fj/8+GhWlOxl2uvrwi5HRNJYomHxb8By4GvALUTvpv5yUEXJRyac1JuPjenDAy+uZs22/WGX\nIyJpKtGwyAQecPdr3P1q4EHgiFlgJRh3TTyJ3HYZfPOJJVooSURCkWhYvAS0r/e6PdHJBKUV5Hdq\nx3evGsXCDbt49K0NYZcjImko0bDIcfdD50Biz3ODKUkac/UpfTl/RD73PreSTbtSbh0oEUlyiYZF\nmZkdmiLczMYCWtqtFZkZP7p6NAb8pxZKEpFWlmhY3AL8xczeMLPXiS6ROjW4sqQx/brl8s0rTuD1\n90qZ+fbmsMsRkTSSaFgMBk4h+qmoF4BVNL3qnQTo02cMZOzAbtz99LuU7qsIuxwRSROJhsV33H0v\n0BW4FJgG/DKwqqRJkYhx73VjOFhVw12ztFCSiLSORMOibja7jwG/cvengOxgSpJ4huZ35JaLh/PM\n0hKeW7Y17HJEJA0kGhabY2twXw/MMbN2zThWAnDT+CGM6tOZ7zy1jD0HqsIuR0RSXKJ/8K8nujzq\n5e6+G+gO3B5YVRJXVkaEn1w3hp1llfzXnBVhlyMiKS7R9SwOuPtMd18de13i7s8HW5rEM7pvF24a\nP4Q/L9jI39ZsD7scEUlhOpXUxt1y8XCG9OjAHTOXcKCyOuxyRCRFKSzauJysDO65dgwbdx7kp8+/\nF3Y5IpKiFBYpYNzg7nzmzIH8/m/v8/YHu8IuR0RSkMIiRXzj8pH06ZzDN2csoaJa63aLSMtSWKSI\nTjlZ/Ojqk1m9bT+/eGVt2OWISIpRWKSQC0/oydWn9OUXr65h5da9YZcjIilEYZFivnPVKDrnZPHN\nGUuo0UJJItJCFBYppnuHbO4qPonFm/bwv397P+xyRCRFKCxS0FVj+nDJib247/lVbNhRFnY5IpIC\nFBYpyMz44cdHkxWJcMcTS7VQkogcN4VFiurdJYf//NiJzFu3gz/P3xh2OSLSxiksUtjk0/tz1pA8\nfvTMCrbuKQ+7HBFpwxQWKczM+PE1J1NVW8udT2rdbhE5dgqLFDeoRwf+49KRvLjiQ55ZWhJ2OSLS\nRiks0sCN5wyisF8XvvfUcnaVVYZdjoi0QQqLNJCZEeHe68aw52AVP3j63bDLEZE2SGGRJk7o3Zmv\nXDiMme9s5pVV28IuR0TaGIVFGvnqhUMZ3rMj3565lP0VWihJRBKnsEgj7TIzuPe6MZTsLecnz60M\nuxwRaUMCDQszu9zMVpnZGjO7o5H995vZotjjPTPb3WB/ZzPbbGb/E2Sd6eTUAd248ezBPDJvA/98\nf2fY5YhIGxFYWJhZBvAQcAUwCphiZqPqt3H3W929yN2LgJ8DMxu8zQ+A14KqMV3ddtkI+nVrzx1P\nLKG8SgsliUh8QY4sxgFr3H2du1cC04FJR2k/BXis7oWZnQb0Ap4PsMa0lJudyT3XjGHd9jIefGl1\n2OWISBsQZFj0BepPSrQptu0IZjYQGAy8HHsdAX4K3H60b2BmN5nZAjNbUFpa2iJFp4tzh/fg+rH9\n+PXr61i2eU/Y5YhIkgsyLKyRbU3NNzEZmOHudedEvgLMcfejzoDn7tPcfay7j83Pzz+OUtPTt68c\nRfcO2XxjxhKqamrDLkdEkliQYbEJ6F/vdT9gSxNtJ1PvFBRwFjDVzNYD9wE3mNk9QRSZzrrkZvGD\nSaN5t2Qvv3ljXdjliEgSCzIs5gPDzWywmWUTDYRZDRuZ2UigGzCvbpu7f8rdB7j7IOA24BF3P+LT\nVHL8Lh/dmytG9+ZnL65mben+sMsRkSQVWFi4ezUwFZgLrAAed/flZna3mRXXazoFmO6aEjU03590\nEjmZEe54Ygm1WrdbRBphqfI3euzYsb5gwYKwy2iz/rJgI7fPWMIPJp3EZ84aFHY5ItJKzGyhu4+N\n1053cAsA153Wj/OG9+CeZ1eyeffBsMsRkSSjsBAgulDSf119Mg58+69at1tEDqewkEP6d8/l9stG\n8uqqUp5ctDnsckQkiSgs5DA3nDWIUwd05fuz32X7/oqwyxGRJKGwkMNkRIx7rx3DgYoa7pq1POxy\nRCRJKCzkCMN7deLmi4bx9JISnl++NexyRCQJZIZdgCSnL50/lGeWlnDnk8twYFjPjgzsnktmhv5/\nIZKOFBbSqOzMCP99XSGTp83jS48uBCArwxiU14FhPTseegzNjz7aZ2eEXLGIBElhIU06uV8X/vHt\nS1izbf9hj5Vb9zF3+VbqbvY2g75d20cDJL/jYWHSNTc73B9CRFqEwkKOqmO7TIr6d6Wof9fDtldU\n17B++4GPQqQ0+nXe2h1UVH80g22PjtkMbRAgw3p2pHfnHMwam5hYRJKRwkKOSbvMDEb27sTI3p0O\n215T62zedZA1pfsOG43MXryFveXVh9p1bJfJ0PwODK0LkFigDNB1EZGkpLCQFpURMQbk5TIgL5eL\nTuh1aLu7U7q/gjXb9rO23mjk72t2MPPtj24AzM6IMKhHLsN6duSsIXn8yxkDyYhoBCISNoWFtAoz\no2enHHp2yuHsoT0O27evvIq1pWWHjUSWb9nLnKVbmfnOZu77RCFD8zuGVLmIgGadlSTl7sxavIXv\nPrWc8qoabr9sJDeeM1ijDJEWpllnpU0zMyYV9eWFW8dz7rAe/PCZFUyeNo/128vCLk0kLSksJKn1\n7JzDbz87lvs+UcjKrfu44oE3+MPf12uRJpFWprCQpGdmXHdaP56/dTzjBnfne7OW86nf/oONOw+E\nXZpI2lBYSJvRp0t7Hr7xdO699mSWbt7D5T97nf/7xwatvSHSChQW0qaYGZ88fQDPff08igZ05dt/\nXcYNv/8nW7S6n0igFBbSJvXrlssfv3AGP/z4aBZu2MVl97/O4/M3apQhEhCFhbRZZsanzxzI3K+P\n56S+nfnGE0v4/MPz2bqnPOzSRFKOwkLavP7dc/nTv57JXRNHMW/dDibc/xoz396kUYZIC1JYSEqI\nRIzPnTOY524Zz4henfj3xxfzxUcWsm2fRhkiLUFhISllUI8O/PlLZ3Hnx07kjdWlTLj/dWYt3qJR\nhshxUlhIysmIGP963hCe+dp5DMrrwNcee4ev/N/bbN9fEXZpIm2WwkJS1rCeHZnx5bO444oTeGnF\nNibc/zpzlpaEXZZIm6SwkJSWmRHhy+cP5emvnUvfru35yv+9zc2PvcOussqwSxNpUxQWkhZG9OrE\nzK+czW0TRvDcshIuvf91nl++NeyyRNoMhYWkjayMCFMvGs6sqefSs1M7bnp0If/+50XsOVAVdmki\nSU9hIWnnxD6defKr53DLxcOZtXgLE372Gq+s3BZ2WSJJTWEhaSk7M8Ktl47gya+eQ9f22dz48Hy+\nMWMxe8s1yhBpjJZVlbQ2um8XZt18Dg++tJpfvrqWN1Zv595rx3DusB5NHhPvjo1493QkcsdHIreF\nRCw65UndV5EgaVlVkZhFG3dz218Ws2bb/rBLOSZmYEDELPrc7LDXh7YTveM9XluASCS2rd6+7MwM\n2mVGyMmK0C4zg5ysCDlZddsa/9ouzv6GXzMzdNKjtSS6rKpGFiIxRf278vTN5/L4go3sKjv66ah4\n/5GP9//8uMfHaeDuuEOtg+PUOuB+2Gv3WDugtjb2NXacN9XWY21ibYm9jraNvk9FdS0V1TVUVNWy\n+0Al5VXR1/W/llfXJDQ6akpmxI4IkS65WQzonsvA7rkMyOvAwLzo8/xO7TSyagUKC5F6crIyuOGs\nQWGX0ea5O1U1TnksVMqraqio/uhrRYPXR/taF0I7yypZsH4Xsxdvof6quu2zMhjQPZcBsfAYmBcL\nk+659O3WniyNUlqEwkJEWpyZkZ1pZGdGIKdl37uyupZNuw6wYecBNu48wIYddY8y3lhdSnlV7aG2\nGRGjoGsOA7t3YEBebr2RSS4D8zrQsZ3+BCZK/1Ii0qZkZ0YYkt+RIfkdj9hXW+uU7q84FB4f1IXJ\nzgM8u7SEXQ3uqcnrkH1oRFI3GomOTHLJ76jTW/UpLEQkZUQiRq/OOfTqnMO4wd2P2L+3vIoP6kYi\nO8sOPZ+/fhdPLd5y2HWW3OzY6a0Gp7YG5uVS0DX9Tm8FGhZmdjnwAJAB/Nbd72mw/37gwtjLXKCn\nu3c1syLgl0BnoAb4kbv/OchaRST1dc7JYnTfLozu2+WIfRXVNWzadTAWIGVs2HmAD3YcYN32Ml59\nr5TK6sNPb/Xt2j4aIrEAGZrfkfEj8lM2RAILCzPLAB4CLgU2AfPNbJa7v1vXxt1vrdf+ZuCU2MsD\nwA3uvtrMCoCFZjbX3XcHVa+IpLd2mRkMze/I0CZOb324r5wNO6IB8sHOA7EwKeOZpSXsjp3e6t+9\nPbdcPIKPFxWk3Md/gxxZjAPWuPs6ADObDkwC3m2i/RTgewDu/l7dRnffYmbbgHxAYSEirS4SMfp0\naU+fLu05c0jeEfv3HKzin+/v5IGX3uO2vyzmF6+s4ZZLhjNxTAGRSGpc9wgy+voCG+u93hTbdgQz\nGwgMBl5uZN84IBtY28i+m8xsgZktKC0tbZGiRUSaq0v7LC4d1YvZU8/lV58+jayMCLdMX8QVD7zB\nc8tKUmKlxiDDorE4bepfbDIww91rDnsDsz7Ao8CN7l7b8CB3n+buY919bH5+/nEXLCJyPMyMy0f3\n5tlbzuPBKadQVVvLl//4Nlf9/E1eXvlhmw6NIMNiE9C/3ut+wJYm2k4GHqu/wcw6A88Ad7r7W4FU\nKCISgEjEKC4s4Pmvj+e+TxSyt7yKzz+8gKt/8XfeXL29TYZGkGExHxhuZoPNLJtoIMxq2MjMRgLd\ngHn1tmUDfwUecfe/BFijiEhgMjMiXHdaP17+jwv48TUns21vOZ/+3T/45LS3+Me6HWGX1yyBhYW7\nVwNTgbnACuBxd19uZnebWXG9plOA6X541F4PjAc+Z2aLYo+ioGoVEQlSVkaEKeMG8MrtF/D94pN4\nf3sZn5z2Fp/53T9454NdYZeXEM06KyLSyg5W1vDHtzbwy9fWsrOskotP6Mmtl45o9P6PoCU666zC\nQkQkJGUV1Tz89/X8+rW17C2v5orRvbn10hGM6NWp1WpQWIiItBF7Dlbxuzff5/dvvk9ZZTUTxxTw\n9UuGNzr/VUtTWIiItDG7yiqZ9sY6Hv7beiqqa7jm1H7ccvFw+nfPDex7KixERNqo0n0V/Oq1tTz6\n1gZqa53rT+/P1AuHUdC1fYt/L4WFiEgbt3VPOQ+9sobp8z/AMP7ljAF85cKh9OzUcouEKCxERFLE\npl0H+J+X1/CXhZvIyjA+e9YgvnT+ULp3yD7u91ZYiIikmPXby3jwpdX8ddFmcrMyuPGcwXzxvCF0\nyc065vdUWIiIpKg12/Zx/4ureWZJCZ1yMrnpvCFMvWjYMa3sl2hYpNaE6yIiaWBYz0489C+nMudr\n53HmkDyWb9kb+BKwWlZVRKSNGlXQmd/cMJaqmiMm5W5xGlmIiLRxrbGUq8JCRETiUliIiEhcCgsR\nEYlLYSEiInEpLEREJC6FhYiIxKWwEBGRuFJmug8z2wOsbmRXF2BPAtt6ANsDKC2exmpprfdJ9Jh4\n7Zra35ztydQnEF6/BN0nR9ui5MiLAAADiElEQVSX7P2S7L8rx9MmzD4Z6O75cVu5e0o8gGmJbm9i\n24Jkqrs13ifRY+K1a86/fVvokzD7Jeg+acv9kuy/K8fTpi30SSqdhprdjO1NtQ1DS9VyLO+T6DHx\n2jXn376p7cnUJxBevwTdJ0fbl+z9kuy/K8fTJun7JGVOQx0vM1vgCcy8KK1HfZKc1C/JpzX6JJVG\nFsdrWtgFyBHUJ8lJ/ZJ8Au8TjSxERCQujSxERCQuhYWIiMSlsBARkbgUFo0wsw5m9gcz+42ZfSrs\neiTKzIaY2e/MbEbYtUiUmX089nvylJlNCLseiTKzE83sV2Y2w8z+rSXeM23Cwsx+b2bbzGxZg+2X\nm9kqM1tjZnfENl8DzHD3LwLFrV5sGmlOv7j7Onf/QjiVpo9m9smTsd+TzwGfDKHctNHMflnh7l8G\nrgda5CO1aRMWwMPA5fU3mFkG8BBwBTAKmGJmo4B+wMZYs5pWrDEdPUzi/SKt42Ga3yd3xvZLcB6m\nGf1iZsXAm8BLLfHN0yYs3P11YGeDzeOANbH/sVYC04FJwCaigQFp9G8Uhmb2i7SC5vSJRd0LPOvu\nb7d2remkub8r7j7L3c8GWuRUerr/IezLRyMIiIZEX2AmcK2Z/ZLkmu4gXTTaL2aWZ2a/Ak4xs2+F\nU1raaup35WbgEuA6M/tyGIWluaZ+Vy4wswfN7NfAnJb4Rpkt8SZtmDWyzd29DLixtYuRQ5rqlx2A\n/iCFo6k+eRB4sLWLkUOa6pdXgVdb8hul+8hiE9C/3ut+wJaQapGPqF+Sj/okObVav6R7WMwHhpvZ\nYDPLBiYDs0KuSdQvyUh9kpxarV/SJizM7DFgHjDSzDaZ2RfcvRqYCswFVgCPu/vyMOtMN+qX5KM+\nSU5h94smEhQRkbjSZmQhIiLHTmEhIiJxKSxERCQuhYWIiMSlsBARkbgUFiIiEpfCQkRE4lJYiIhI\nXOk+kaBIoMzsJOABYADwKNATeMTd54damEgz6Q5ukYCYWQ7wNvAJYB2wEljo7teEWpjIMdDIQiQ4\nlwDv1M3VE5vo7afhliRybHTNQiQ4pxAdWWBmBcB+d/9buCWJHBuFhUhwKvhoed4fA9kh1iJyXBQW\nIsH5EzDezFYBi4F5ZvazkGsSOSa6wC0iInFpZCEiInEpLEREJC6FhYiIxKWwEBGRuBQWIiISl8JC\nRETiUliIiEhcCgsREYnr/wHfOXtMJ449MwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1bce298dbe0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#测试不同的α值对预测性能的影响\n",
    "def test_lasso_alpha(*data):\n",
    "    X_train, X_test, y_train, y_test,train,test = data\n",
    "    alphas = [1, 2, 5, 10, 20, 50, 100, 200, 500, 1000]\n",
    "    scores = []\n",
    "    for i, alpha in enumerate(alphas):\n",
    "        lassoRegression = linear_model.Lasso(alpha=alpha)\n",
    "        lassoRegression.fit(X_train, y_train)\n",
    "        scores.append(lassoRegression.score(X_test, y_test))\n",
    "    return alphas, scores\n",
    "\n",
    "def show_plot(alphas, scores):\n",
    "    figure = plt.figure()\n",
    "    ax = figure.add_subplot(1, 1, 1)\n",
    "    ax.plot(alphas, scores)\n",
    "    ax.set_xlabel(r\"$\\alpha$\")\n",
    "    ax.set_ylabel(r\"score\")\n",
    "    ax.set_xscale(\"log\")\n",
    "    ax.set_title(\"Ridge\")\n",
    "    plt.show()\n",
    "    \n",
    "alphas, scores = test_lasso_alpha(X_train, X_test, y_train, y_test,train,test)\n",
    "show_plot(alphas, scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\coordinate_descent.py:491: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.\n",
    "  ConvergenceWarning) ？不太懂是什么意思……"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
